Earlier, I looked at the observed performance spread that we see in NHL shooting. When I talk about shooting talent, I mean a player's ability to score from a particular spot on the ice relative to the league average. To that end, I looked at player shooting at even-strength over the last five seasons, using only road shots to reduce scorer bias.
But, of course, when we measure a player's performance, we're only getting a snapshot of his ability. The range of ability is usually much smaller than the range of performance. So to determine the range of player ability, I started out assuming that the talent distribution matched the observed performance ability. I then updated my estimate of each individual player's range of abilities using the actual outcomes that player achieved in each season.
For Ilya Kovalchuk, who I believe to be an exceptional shooter, the progression of his mean talent estimate over the course of five seasons looked like:
Initial estimate: 6.38% (league average)
2005-06: 8.71
2006-07: 8.50
2007-08: 10.01
2008-09: 10.69
2009-10: 10.63
I ran this for every player in the league over five seasons (for season where they had fewer than 20 shots, I didn't update my estimate) and ended up with the following estimated distribution of talent:
Just to recap – I started out by assuming that the player talent distribution looked like the blue curve, and updated each player's individual distribution based on how he did over the course of five seasons. The green curve shows the distribution of mean estimated talent across the league by this method. So clearly my initial estimate was wrong – I started out with a distribution that had an average of 6.38% and a standard deviation of 1.53%, while using that distribution to start implies a significantly narrower distribution of talents.
How do we fix this? Well, I re-ran the problem, using my green curve as the initial estimate of talent distribution. I iterated until the final talent distribution matched the initial talent distribution (and was stable for several successive iterations). The final curve had a standard deviation of 0.44% (the mean was the same, obviously). The implication is that the distribution of shooting talent is about 71% narrower than the observed talent distribution, which is – coincidentally or not – almost the same answer as I got (74%) when I looked at the relationship between shooting performance in odd and even shots.
As for Kovalchuk, I'm afraid I don't have such a good answer. He goes from being roughly 2.8 standard deviations from the mean with the original distribution to being somewhere between 7-8 in the final estimate. I'm comfortable with Kovalchuk in the range of three standard deviations above the mean (eg – Tanguay may be just as good, so he's clearly not 1-in-1000) but eight is clearly not the right answer.
