Shootouts: Goaltender True Talent

As an add-on to my previous piece in shooter talent, I wanted to apply the same methodology to goaltender performance on shootouts. First, let's see if performance in 2005-06 predicted future performance. I selected at the 26 goaltenders who faced at least 8 shots in both 2005-06 and 2006-07, divided them into groups with above-average and below-average save percentage, and looked at their total save percentage over the next three seasons:

	2005-06	2006-07	2007-08	2008-09
Above-Average	57/231	107/370	79/274	68/212
Shot % Against	25	29	29	32
Below-Average	98/227	91/279	100/306	53/183
Shot % Against	43	33	33	29

Going forward, the below-average group gave up goals on 32% of opponent shots. The above-average group was at 30%. So the gap between the two groups dropped from 18% to 2%. We expect the above-average group to regress to the mean somewhat, but why does the "below-average" group see such a huge improvement - in fact, all the way to league average?

Again, as with shooters, it's selection bias. I picked goaltenders who played in the "before" part of my study and the "after" part. That means they were good enough in the rest of their game to keep a regular NHL job, so I essentially assembled a group that consisted of the 14th-26th best starting goaltenders in the league, which are guys clustered around the league average. So it's no surprise that they come out average in future years.

What if we compare the first two post-lockout seasons to the second two? Again, we have 26 goalies:

	05-06/06-07	07-08/08-09
Above Average	171/653	180/589
Shot % Against	26	31
Below Average	176/426	179/554
Shot % Against	41	32

This doesn't look substantially different! What if we compare the first three seasons to 2008-09? This time we have 28 goaltenders:

	05-06 to 07-08	2008-09
Above Average	244/899	82/294
Shot % Against	27	28
Below Average	289/734	87/265
Shot % Against	39	33

It doesn't matter how we slice the data, we get roughly the same result. At the absolute most, a top ten goaltender is five points better in the shootout than an average goalie (28% vs 33%). Over an average season - 10 shootouts or 33 shots - the 28% goalie makes a .500 team into a .570 team. That is worth 0.7 points in the standings, or 0.35 wins. That's perhaps 10% of a top goaltender's total value above average, so it's not insignificant, but given that the best performer so far is Johan Hedberg, who's well below-average in the thousands of minutes he plays during regulation play, shootout ability is unlikely to drive a goaltending decision.