Filed under:

# Beating Save Percentage to Death Part 3: Goaltenders

Goalie differences

Do goalies differ? As a fan of the Quebec Nordiques, my answer is an emphatic "Yes!" I would bet fans of the pre-lockout Ottawa Senators would wholeheartedly agree.

Not everybody does. Sunny Mehta recently said "No one has conclusively shown a meaningful difference in skill between NHL goaltenders. I'm not saying it doesn't exist, I'm just saying no one's really proved it, and that all signs point to goaltending differences being far less important than everyone thinks."

Let's look at even strength save percentages for all goalies from 1998-2010. First, looking at the raw data.

> AnovaModel.1 <- aov(ESVPCT ~ last.first, data=Dataset)

> summary(AnovaModel.1)

Df  Sum Sq  Mean Sq    F value  Pr(>F)

last.first 249 0.70705 0.00283956 3.9338 < 2.2e-16 ***

Residuals  785 0.56665 0.00072185

So goaltenders do differ significantly. The differences between goalies accounts for about 56% of the total variability seen in the raw data.

Correcting for year

I have already shown average even strength save percentage has a small but significant increase over the time period we are looking at. So I have recalculated the model using adjusted even strength save percentages. It always makes me a little nervous when people say they are adjusting the data without describing what they did. On a theoretical level, I have converted the observed save percentage to a standard normal Z-statistic using the observed mean for the year as the mean and the observed standard deviation over the 12 year period as the standard error. I have then rescaled the Z-statistics to the 2009-2010 mean save percentage of 0.919. From a practical standpoint, this amounts to adding or subtracting a small constant to each observation. For example, in 1997 the observed average save percentage was 0.915. To make the adjustment, I added 0.004 to each observation from 1997. The results:

> AnovaModel.2 <- aov(ADJESSVP ~ last.first, data=Dataset)

> summary(AnovaModel.2)

Df  Sum Sq  Mean Sq    F value  Pr(>F)

last.first 249 0.69835 0.00280463 3.8826 < 2.2e-16

Residuals  785 0.56705 0.00072236

So basically the same. Goaltenders do differ significantly. The differences between goalies accounts for about 55% of the total variability seen in the adjusted data.

Regular Goalies

Finally, I removed all goalies who had played less than 10 games in a season from the data.

> AnovaModel.1 <- aov(ADJESSVP ~ last.first, data=Dataset2)

> summary(AnovaModel.1)

Df  Sum Sq   Mean Sq    F value Pr(>F)

last.first 165 0.046939 0.00028448 1.6446 1.344e-05

Residuals  595 0.102922 0.00017298

A little less dramatic, but regular goaltenders do differ significantly. The differences between regular goalies accounts for about 31% of the total variability seen in the adjusted data.

Conclusion

The difference in goalies over the period 1997-2010 is statistically significant. The difference in goalies who played more than 10 games in a season over the period 1997-2010 is also statistically significant.