I thought I'd join the ever-exciting, ever-controversial shot quality discussion/debate/argument/bloody fued. As I mentioned in the comments section of a post by Gabe, I did the following calculation.
I took the last 4 full seasons, kept only 5-on-5 situations with both goalies on the ice. For each team, I split the games into odd games and even games. I then found the correlation between goals, shots, fenwick, corsi, and weighted shots in odd games, and those same stats in even games.
"Weighted shots" are based on Ken Krzywicki's Shot Quality model from 2008-2009, which included distance, shot type, rebounds, EV/PP/SH, and shot after giveaway. It did not include (x,y) coordinates or shot angle, which Ken did the next year, and which Schuckers used in his DIGR stuff. I looked only at team offense, and used per 60 minute stats. Below, "odd GoalsF" means Goals For per 60 minutes in odd games. Here is the correlation matrix:
| even GoalsF | even WShotF | even ShotsF | even FenwickF | even CorsiF | |
| odd GoalsF | 0.28 | 0.37 | 0.31 | 0.28 | 0.29 |
| odd WShotF | 0.33 | 0.66 | 0.53 | 0.53 | 0.53 |
| odd ShotsF | 0.42 | 0.57 | 0.72 | 0.73 | 0.70 |
| odd FenwickF | 0.38 | 0.54 | 0.69 | 0.76 | 0.75 |
| odd CorsiF | 0.40 | 0.59 | 0.70 | 0.78 | 0.83 |
So, for example, this says that the correlation between Goals For per 60mins in even games (even GoalsF) and Shots For per 60mins in odd games (odd ShotsF) is 0.42. A couple observations:
1. The first column is various stats from odd games and Goals For in even games
| even GoalsF | |
| odd GoalsF | 0.28 |
| odd WShotF | 0.33 |
| odd ShotsF | 0.42 |
| odd FenwickF | 0.38 |
| odd CorsiF | 0.40 |
According to this, weighted shots in odd games are more correlated with goals in even games than goals in odd games are with goals in even games. But shots, Fenwick, and Corsi all have higher correlations than weighted shots.
2. Let's also look at the first row, which is various stats from even games and goals from odd games:
| even GoalsF | even WShotF | even ShotsF | even FenwickF | even CorsiF | |
| odd GoalsF | 0.28 | 0.37 | 0.31 | 0.28 | 0.29 |
Weighted shots from even games have the highest correlation with goals from odd games. At this point, i should say that I'm not making any claims about shot quality, just reporting the results of what I've done. Please don't shoot the messenger. The correlation happens to be higher in this case.
If we do something really crude (and possibly statistically illegal) and average these last two sets of correlations, we get
| GoalsF | WShotF | ShotsF | FenwickF | CorsiF |
| 0.28 | 0.35 | 0.37 | 0.33 | 0.35 |
I think it's too close to call from what I have done. It seems like weighted shots, shots, Fenwick, and Corsi are all higher than goals, but it's hard to tell which is the highest. To be more confident about these correlations, it would be better to split the data randomly, compute the correlations, and repeat, like JLikens did here. I did not split the data in half randomly, nor did I repeat. I only split into odd and even games and computed the correlations once. But I imagine the results wouldn't change a whole lot, or at least not enough to make me want to say anything conclusive about shot quality being significantly better or worse than shots, Fenwick, or Corsi. It would probably be better to look at something other than correlations before making such a claim anyway.
Also, in my opinion, it doesn't really matter if shot quality is better or worse than shots/Fenwick/Corsi. It doesn't have to be better than those to be useful. If shots/Fenwick/Corsi coupled with shot quality is more useful than shots/Fenwick/Corsi alone, that's good enough for me.
In short, my two cents, which is likely worth less than $0.01, on the shot quality topic is that it is too early to say that Shot Quality is completely useless and hopeless, and it's too early to say that Shot Quality is the greatest thing since sliced bread. My opinion is that the correlations above suggest that it is worth studying shot quality some more. But that is just my opinion.
Btw, the results for goals, shots, Fenwick and Corsi are fairly similar to what JLikens got in his nice study of shots, Fenwick, and Corsi in the post I mentioned before. We did slightly different things though, which probably account for the differences:
– I did not do score-tied (sorry, please don't hate me, I can repeat with score-tied sometime if desired)
– I chose to look only at offense. JLikens looked at offense and defense. So, for example, I looked at GF/60, instead of goal ratio GF/(GF+GA), which is what JLikens used.
– I did not split the data in half randomly, repeating this many times. I used odd and even games, and compute correlations once.
