A number of esteemed hockey analysts have attacked this problem, and I thought I would look at it from the same perspective as my shooting analysis. To eliminate scorer bias and man-advantage situations, I looked only at even-strength save percentage on the road over the last three seasons. What's important here is to pick the right threshold for the minimum number of shots faced – you can see how the skill component changes as I increase it:
| # Shots | 0 | 50 | 100 | 250 | 500 | 1000 |
| Skill % | 36.9 | 28.6 | 26.2 | 38.5 | 16.1 | 0.0 |
By the time you get to regular starting goaltenders, there’s very little to distinguish them. Some of this is legitimate skill – goaltenders who play every day tend to both 1) be better than other goaltenders; 2) post good results. So we have a narrowing of skill and a selection bias that narrows the outcomes. You can make a compelling argument that any one of Roberto Luongo, Tomas Vokoun, Jonas Hiller or any one of a number of other goalies is the best in the league; their performance variation from year-to-year due to luck alone is surely larger than their inherent skill difference.
So let’s take 250 shots as the minimum for constructing our ability distribution. If we assume every goaltender sits on that distribution and update it with three years of performance data – assuming skill is constant over three seasons – then we can estimate true talent. One of the biggest gaps in the league is between Roberto Luongo, Olympic Gold Medalist and Michael Leighton, obvious replacement-level goaltender (don’t tell Paul Holmgren):
That red line is league-average. Using this methodology, we are about 90% confident that Luongo is better than Leighton – if we added bias-corrected home data to the analysis, we'd be even more confident. The difference between the two goaltenders is about 2.3 wins over the course of a season, which is an underestimate, but not by much.
Overall, Luongo is regressed roughly 33% to the mean relative to his actual performance over the last three seasons. if we used home and road data, we'd end up regressing him very little – the takeaway here is that if you want to estimate a goaltender's true talent, you may as well just look at his even-strength road save percentage.
