Replacement goaltenders are not the same as threshold goaltenders.
In baseball, VORP, or Value Over Replacement Player, is one metric used to assess the value of players. Keith Woolner, the creator of VORP, says if a team has to replace a starting player "the team may be stuck playing a utility bench player, bringing up a AAA player, or finding some other journeyman to fill the gap." Woolner continues "Replacement level is the *expected* (his emphasis) level of performance the average team can obtain if it needs to replace a starting player at minimal cost."
I looked for replacement goaltenders by pulling out all goalies who played in 5 games or fewer in a single season.
|
Number |
Even Strength |
Penalty Kill |
Power Play |
Overall |
|
|
1 game |
78 |
0.897 |
0.821 |
0.900 |
0.884 |
|
1-2 games |
120 |
0.897 |
0.840 |
0.864 |
0.885 |
|
1-3 games |
155 |
0.895 |
0.852 |
0.878 |
0.887 |
|
1-4 games |
181 |
0.897 |
0.852 |
0.880 |
0.887 |
|
1-5 games |
207 |
0.898 |
0.851 |
0.858 |
0.887 |
These values probably represent a slight overestimate of true replacement level performance, as this includes some injured starters (Khabibulin, Fernandez), and some top prospects (Bernier, Schneider), as well as some “one-hit wonders” (the locally famous Richard Shulmistra, among others).
By comparison, NHL average and threshold values are:
|
Even Strength |
Penalty Kill |
Power Play |
Overall |
|
|
NHL Average |
0.917 |
0.868 |
0.914 |
0.909 |
|
Calculated Threshold |
0.914 |
0.863 |
0.911 |
0.905 |
Obviously, the value you pick for threshold makes a tremendous difference in calculated GVT. The average NHL team in 2009 faced 2478 shots, 1971 at even strength, 437 on the penalty kill, and 70 on the power play. If a hypothetical starter played 2/3 of his teams minutes, I calculate the goaltender portion of GVT (GGVT) of 4.16 using the threshold numbers and 24.40 using my replacement numbers (As always, I am not attempting to calculate puck handling or shootout, so I am not calculating total GVT). Similarly for Ryan Miller and Vesa Toskala.
|
Name |
GGVT threshold values |
GGVT replacement values |
|
Ryan Miller |
35.3 |
61.52 |
|
Vesa Toskala |
-14.06 |
-4.65 |
|
NHL Average |
4.16 |
24.4 |
Toskala remains minus even with my values because he remains -7.50 on the penalty kill. Given my other data showing a lack of goalie effect on the PK, I’m not sure it is fair to attribute PK success or failure to individual goaltenders. If you take out the PK, Toskala is a +2.85 in the other situations.
Conclusion
Relative to observed replacement goaltender performance, the standard method of calculating threshold save percentage severely overestimates the ability of a replacement level goaltender to stop the puck at even strength, on the power play, and overall. Since GVT values depend on this threshold performance, GVT values calculated this way will greatly underestimate the magnitude of goaltender contribution relative to replacement level goaltenders.
Acknowledgments
Again, I want to thank Sunny Mehta for his very helpful correspondence. I also want to thank Tom Awad for his patient and thoughtful discussions of GVT.
