Recently I looked at goal reliability for forwards and defensemen. Today I'll follow that up with the rest of the traditional stats (A, Pts, +/-, PIM) for forwards, and briefly talk about how this applies to individual players through regression to the mean.
Methods See previous posts
|Stat||Games when alpha > 0.707||95% Confidence Interval||alpha at 120 games|
Looking back at the first article I wrote, I took F G reliability at 0.7 instead of 0.707. This actually extends it 2 games to 40 games. Interesting to see here that even at 120 games the +/- stat never stabilizes, reaching a 0.447 at 120 games.
The basis of this study is rooted in True Score Theory. In essence were looking at how many games until a stat is mostly skill, and less random noise. The 0.707 cutoff is somewhat arbitrary, but gives a mark at which to compare stats. It has been used by Pizza cutter in a well referenced study in baseball sabermetrics, although another baseball sabermetrician, Tom Tango, makes a good case for 0.5 in numerous articles here, here, and here. I think the important thing to consider, and routinely talked about by MGL over at inside the book, is that the cutoff doesnt mean that the stat is worthless until we reach 0.707, and that it is completely reliable after. We are in fact working on a continuum based on a population average. (in this case NHL players that have played > 3 years of >20 games). In the case of this study; as games played increases, stat reliability increases. The rate which this occurs gives us information about the reliability of that stat.
Regression to the Mean
A lot has been said about regression to the mean. I think the best article as it relates to reliability was written by a blogger named Eli over at Count the Basket (who has left the blogosphere to work for an NBA team). He explains in great detail the foundation of reliability statistics, as well derivations of popular reliability mathematical formulas. I'll sum up briefly what Eli talks about extensively in his article. Combining the reliability findings with the concept of regression to the mean gives us an idea of the true talent of a player. For example, the number of goals a player records in his first few games is likely to be an unreliable measurement of the true number of goals/games a will score in his career. The best way to account for this would be to weight the player's goals/game with the NHL average. We scale this measurement across games, increasing the player's rate (aka PlayerObsRate, which is goals/game in this case), while reducing the weight of the NHL average rate (aka PopMeanRate). The weighting is dictated by our reliability statistic.
As Eli outlines in his article the formula for the regressed rate (ie. The best estimate of a player's true talent) is PopMeanRate + r*(PlayerObsRate - PopMeanRate). Let's take SJ Shark's Joe Pavelski's goal totals from last season as an example.
For the first 10 games Joe scored 6 goals. League average was 1.978. Our calculated r = 0.50. So we regress Joe's goals by 50% to the mean (the 1-r value). 1.978 + 0.50*(6 - 1.978) = 3.989. At 40 games Joe scored 11 goals, league average 8.397, with r = 0.707. So we regress Joe stat's 29.3% (which is 1-r) to the mean; 8.397 + 0.707*(11 - 8.397) = 10.237.
Interestingly Tom Tango found that r could be calculated to a reasonable degree. His formula is r = PlayerObsRate / (PlayerObsRate + Constant). The constant must be calculated from some previous r value. The fit isn't perfect though it gives a reasonable estimate.
When I have some more time I'll be looking at the rest of the traditional stats for defensemen.