FanPost

SSW: Forward G, A, Pts, +/-, PIM reliability and Regression to the mean

Recently I looked at goal reliability for forwards and defensemen.  Today I'll follow that up with the rest of the traditional stats (A, Pts, +/-, PIM) for forwards, and briefly talk about how this applies to individual players through regression to the mean.

Methods See previous posts

Results

                        F_2btrads_medium

 Stat   Games when alpha > 0.707   95% Confidence Interval   alpha at 120 games 
 Goals 40 27-51 0.860
 Assists 25 18-34 0.911
 Points 14 10-19 0.938
 +/-  n/a n/a 0.447
 PIM 36 29-47 0.907

 

Looking back at the first article I wrote, I took F G reliability at 0.7 instead of 0.707. This actually extends it 2 games to 40 games.  Interesting to see here that even at 120 games the +/- stat never stabilizes, reaching a 0.447 at 120 games.    

Discussion

The basis of this study is rooted in True Score Theory.  In essence were looking at how many games until a stat is mostly skill, and less random noise.  The 0.707 cutoff is somewhat arbitrary, but gives a mark at which to compare stats. It has been used by Pizza cutter in a well referenced study in baseball sabermetrics, although another baseball sabermetrician, Tom Tango,  makes a good case for 0.5 in numerous articles here, here, and here.  I think the important thing to consider, and routinely talked about by MGL over at inside the book, is that the cutoff doesnt mean that the stat is worthless until we reach 0.707, and that it is completely reliable after. We are in fact working on a continuum based on a population average. (in this case NHL players that have played > 3 years of >20 games).  In the case of this study; as games played increases, stat reliability increases. The rate which this occurs gives us information about the reliability of that stat.

Regression to the Mean

A lot has been said about regression to the mean.  I think the best article as it relates to reliability was written by a blogger named Eli over at Count the Basket  (who has left the blogosphere to work for an NBA team).  He explains in great detail the foundation of reliability statistics, as well derivations of popular reliability mathematical formulas.  I'll sum up briefly what Eli talks about extensively in his article.  Combining the reliability findings with the concept of regression to the mean gives us an idea of the true talent of a player.  For example, the number of goals a player records in his first few games is likely to be an unreliable measurement of the true number of goals/games a will score in his career. The best way to account for this would be to weight the player's goals/game with the NHL average.  We scale this measurement across games, increasing the player's rate (aka PlayerObsRate, which is goals/game in this case), while reducing the weight of the NHL average rate (aka PopMeanRate).  The weighting is dictated by our reliability statistic.  

As Eli outlines in his article the formula for the regressed rate (ie. The best estimate of a player's true talent) is PopMeanRate + r*(PlayerObsRate - PopMeanRate).  Let's take SJ Shark's Joe Pavelski's goal totals from last season as an example.

For the first 10 games Joe scored 6 goals.  League average was 1.978.  Our calculated  r = 0.50.  So we regress Joe's goals by 50% to the mean (the 1-r value). 1.978 + 0.50*(6 - 1.978) = 3.989. At 40 games Joe scored 11 goals, league average 8.397, with r = 0.707. So we regress Joe stat's 29.3% (which is 1-r) to the mean;  8.397 + 0.707*(11 - 8.397) = 10.237.

Interestingly Tom Tango found that r could be calculated to a reasonable degree.  His formula is r = PlayerObsRate / (PlayerObsRate + Constant). The constant must be calculated from some previous r value.  The fit isn't perfect though it gives a reasonable estimate.  

When I have some more time I'll be looking at the rest of the traditional stats for defensemen.

If this FanPost is written by someone other than one of the blog's editors, the opinions expressed in it do not necessarily reflect those of this blog or SB Nation.

X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Arctic Ice Hockey

You must be a member of Arctic Ice Hockey to participate.

We have our own Community Guidelines at Arctic Ice Hockey. You should read them.

Join Arctic Ice Hockey

You must be a member of Arctic Ice Hockey to participate.

We have our own Community Guidelines at Arctic Ice Hockey. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9355_tracker