Comments / New

Simulation of GVA and the estimation error of GVA

Sunny Mehta recently suggested that the observed range of goaltender save percentages over the course of a single season could essentially be duplicated by an average goalie and random factors.  It dawned on me that, if this were true, that observed goaltender performance metrics ought to similarly be indistinguishable from metrics derived from an average goaltender plus random factors.  To explore this, I ran some simulations using a pseudo-random number generator.  For these simulations I used an  ES save percentage of 0.917, PK save percentage of 0.868, and PP save percentage of 0.914

The range of GVA seen in a single season can be duplicated by an average goaltender and random factors.  GVA is a fairly imprecise estimate, with a large standard error and a large confidence interval.

Theoretical Average Goaltender Average Workload
The average NHL team in 2009 faced 2478 shots, 1971 at even strength, 437 on the penalty kill, and 70 on the power play.  If a hypothetical starter played 2/3 of his teams minutes, he faced 1314 shots at ES, 291 on the PK, and 47 on the PP.  Running 10,000 simulated seasons gives:
      GVA         
 Min.   :-45.4900 
 1st Qu.: -7.4900 
 Median :  0.5100 
 Mean   :  0.0952 
 3rd Qu.:  8.5100 
 Max.   : 47.5100 
Std. Dev. 11.57677
Thus a 95% CI for GVA is [22.69, -22.69]
Average Goaltender

Ryan Miller

workload
Miller led the league in GVT.  He faced 1690 ES shots, 321 PK shots, and 87 PP shots.  10,000 simulated seasons with an average goaltender facing that workload:
      GVA         
 Min.   :-46.8800 
 1st Qu.: -8.8800 
 Median :  0.1200 
 Mean   :  0.1159 
 3rd Qu.:  9.1200 
 Max.   : 48.1200 
Std. Dev.  13.10482
Thus a 95% CI for GVA at this workload is [25.69, -25.69].  Miller's GVA was 40.12, which is outside the CI, but within the range generated.
Average Goaltender

Vesa Toskala

workload
Toskala, on the other hand, had the worst GVT in the league.  He faced 540 ES shots, 151 PK shots, and 23 PP shots.  10,000 simulated seasons with an average goaltender facing that workload:
      GVA         
 Min.   :-37.3700 
 1st Qu.: -5.3700 
 Median :  0.6300 
 Mean   :  0.0421 
 3rd Qu.:  5.6300 
 Max.   : 30.6300 
Std. Dev.  8.026439
Thus a 95% CI for GVA at this workload is [15.73, -15.73].  Toskala's GVA was -21.37, which is outside the CI, but again within the range generated.
Conclusion
The range of GVA seen in a single season can be duplicated by an average goaltender and random factors.  Extreme values of GVA do lie outside the 95% CI for average GVA.
As an estimate of goaltender performance, GVA is subject to estimation error.  This simulation data suggests that observed GVA is a fairly imprecise estimate, with a large standard error and a large confidence interval.
Acknowledgment
I want to thank Sunny Mehta for his very helpful correspondence.
Addendum
Ryan Miller at Ryan Miller workload
      GVA         
 Min.   :  0.1200 
 1st Qu.: 32.1200 
 Median :  40.1200 
 Mean   : 39.6262
 3rd Qu.:  47.3700 
 Max.   : 83.1200 
Std. Dev.  11.79438
95% CI [63.24, 17.00]
Vesa Toskala at Vesa Toskala workload
      GVA         
 Min.   :-61.3700 
 1st Qu.: -27.3700 
 Median : -20.3700 
 Mean   :  -20.0783
 3rd Qu.:  -14.3700 
 Max.   : 13.6300
Std. Dev.  8.950644
95% CI [-2.54, -37.62]

Looking for an easy way to support Arctic Ice Hockey?
Use our Affiliate Link when shopping hockey merch this holiday season!

Talking Points