Beating Save Percentage to Death Part 3: Goaltenders

Goalie differences

Do goalies differ? As a fan of the Quebec Nordiques, my answer is an emphatic "Yes!" I would bet fans of the pre-lockout Ottawa Senators would wholeheartedly agree.

Not everybody does. Sunny Mehta recently said "No one has conclusively shown a meaningful difference in skill between NHL goaltenders. I'm not saying it doesn't exist, I'm just saying no one's really proved it, and that all signs point to goaltending differences being far less important than everyone thinks."

Let's look at even strength save percentages for all goalies from 1998-2010. First, looking at the raw data.

> AnovaModel.1 <- aov(ESVPCT ~ last.first, data=Dataset)

> summary(AnovaModel.1)

Df  Sum Sq  Mean Sq    F value  Pr(>F)

last.first 249 0.70705 0.00283956 3.9338 < 2.2e-16 ***

Residuals  785 0.56665 0.00072185

So goaltenders do differ significantly. The differences between goalies accounts for about 56% of the total variability seen in the raw data.

Correcting for year

I have already shown average even strength save percentage has a small but significant increase over the time period we are looking at. So I have recalculated the model using adjusted even strength save percentages. It always makes me a little nervous when people say they are adjusting the data without describing what they did. On a theoretical level, I have converted the observed save percentage to a standard normal Z-statistic using the observed mean for the year as the mean and the observed standard deviation over the 12 year period as the standard error. I have then rescaled the Z-statistics to the 2009-2010 mean save percentage of 0.919. From a practical standpoint, this amounts to adding or subtracting a small constant to each observation. For example, in 1997 the observed average save percentage was 0.915. To make the adjustment, I added 0.004 to each observation from 1997. The results:

> AnovaModel.2 <- aov(ADJESSVP ~ last.first, data=Dataset)

> summary(AnovaModel.2)

Df  Sum Sq  Mean Sq    F value  Pr(>F)

last.first 249 0.69835 0.00280463 3.8826 < 2.2e-16

Residuals  785 0.56705 0.00072236

So basically the same. Goaltenders do differ significantly. The differences between goalies accounts for about 55% of the total variability seen in the adjusted data.

Regular Goalies

Finally, I removed all goalies who had played less than 10 games in a season from the data.

> AnovaModel.1 <- aov(ADJESSVP ~ last.first, data=Dataset2)

> summary(AnovaModel.1)

Df  Sum Sq   Mean Sq    F value Pr(>F)

last.first 165 0.046939 0.00028448 1.6446 1.344e-05

Residuals  595 0.102922 0.00017298

A little less dramatic, but regular goaltenders do differ significantly. The differences between regular goalies accounts for about 31% of the total variability seen in the adjusted data.

Conclusion

The difference in goalies over the period 1997-2010 is statistically significant. The difference in goalies who played more than 10 games in a season over the period 1997-2010 is also statistically significant.

Trending Discussions

Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Arctic Ice Hockey

You must be a member of Arctic Ice Hockey to participate.

We have our own Community Guidelines at Arctic Ice Hockey. You should read them.

Join Arctic Ice Hockey

You must be a member of Arctic Ice Hockey to participate.

We have our own Community Guidelines at Arctic Ice Hockey. You should read them.

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.