With Apologies to The Weakerthans...I hate "Shot Quality"
by Hawerchuk on Oct 27, 2011 7:00 AM EDT
in Shot Quality
Email
PrintThe notion of "Shot Quality" is hardly a new one. Alan Ryder probably published his first analysis of it ten years ago and he published his recall of it - because it's hopelessly overwhelmed by arena bias - over four years ago. So what have we learned in ten years?
Alan's initial observation - the likelihood of a shot going in vs a shooter's distance from the net - is a good one. As are adjustments for shot type and rebounds. But it turned out there wasn't much else there. Why? The indispensable JLikens explained why - he put an upper bound on what we could hope to learn from "shot quality" and showed that save percentage was dominated by luck. The similarly indispensable Vic Ferrari coined the stat "PDO" - simply the sum of shooting percentage and save percentage - and showed that it was almost entirely luck. Vic also showed that individual shooting percentage also regressed very heavily toward a player's career averages. An exhaustive search of players whose shooting percentage vastly exceeded their expected shooting percentage given where they shot from turned up one winner: Ilya Kovalchuk...Who proceeded to shoot horribly for the worst-shooting team in recent memory last season.
We started from a truth ("shot quality" does indeed vary on individual shots) and essentially never used that idea to gain any broader insights about individual players or teams. Which team will have the highest even-strength shooting percentage this year? Nobody knows...Or at least nobody is willing to bet on it. What about goaltender talent? Have we come up with a better predictive model for goalie performance because we're able to remove the impact of the defense from overall results?
Now I'm not rejecting finishing and defensive/goaltending talent - they all exist. But I'd like to move past shot quality as such a frequent focus of analysis because I don't think it's going to return much for all of the effort. (Virtually no one entered the shot quality prize competition on this site, so maybe people aren't as interested in it as I think.) If the Edmonton Oilers called you up and offered you their general manager job, what would be your first piece of advice? Would you tell them to chase after players who've had high on-ice shooting percentages over the last few years or who've exceeded their expected shooting percentages given their shot locations?
I think you know the answer. Even though we're nowhere near the level of confidence that baseball analysts have come to, hockey analysis has turned up value in a number of places. There are dozens of things an NHL team should chase before they start looking for finishing talent.
27 comments
|
2 recs |
Do you like this story?
Comments
The similarly indispensable Vic Ferrari coined the stat “PDO” – simply the sum of shooting percentage and save percentage
Actually Oiloblogosphere poster PDO came up with the “PDO” stat. He did it on Vic’s site in August 2008.
Check out the comments in this thread.
When I say ‘coined’, I mean that he coined the name of it. Vic’s been great about giving credit where it’s due.
by Hawerchuk on Oct 27, 2011 9:19 AM EDT via mobile up reply actions
Agreed that this isn’t a matter of Vic not giving PDO credit. It is called “PDO” after all. :)
I have seen a few writers credit Vic with creating it, but I have never seen you, nor Vic do so.
i saw him a few days ago on tyler’s blog complaining about the fact that people attribute PDO to vic and not him. and yes, i am serious.
Driving Play - The Blog with Three First Lines
PDO is actually working for me irl.
He had no idea how often people were using “PDO” and then googled it and was blown away.
I think he just wants his 15min.
Well it hockey stats on the internet, so more like 5min. :)
by Woodguy on Oct 27, 2011 9:50 PM EDT via mobile up reply actions
The piece of shot quality that is (potentially) missing would only be obtained if the equivalent of Pitch f/x hit hockey, capturing velocity and the z-dimension of shot location.
Even then, who knows if it would actually lead to anything of significance.
Blueshirt Banter - Where Rangers' Fans Matter
Tracking the Rangers - Numbers don't lie. They just don't agree with you.
Twitter: RangerSmurf
Seems unlikely anything useful can be done until they put some equivalent of the Fox chip in pucks to accurately track location, velocity, etc.
"[Phil Kessel]'s as streaky as a flipped coin" - Shift
by The '67 Sound on Oct 27, 2011 9:23 AM EDT up reply actions
I would like to see everything tracked electronically. Shooting would be a sliver.
by Hawerchuk on Oct 27, 2011 9:29 AM EDT via mobile up reply actions
It would be the ultimate data breakthrough.
"[Phil Kessel]'s as streaky as a flipped coin" - Shift
by The '67 Sound on Oct 27, 2011 9:40 AM EDT up reply actions
My guess is it’s happening trough a CBA. The last one established something (dunno what) that made the league up the ante on data collection. So I guess we could have something coming in the next year or two.
After all, the technology exists.
I can see it now
Off-ice officials having to take puck back when it goes in the crowd.
Kidding, not only do we have the technology, but we’ve had it so long it’s dirt cheap.
Co-Manager at Arctic Ice Hockey
Want Jets historical stats, Gabe Desjardins metrics, Jets prospect scouting reports, player previews, and old school photos from the WHA days? Get your copy of the First Commemorative Maple Street Press Winnipeg Jets Annual for 2011-12 here.
by Bettman's Nightmare on Oct 27, 2011 11:51 AM EDT up reply actions
Shot Quality would be an awesome name for a band
I write everywhere. You're probably better off following me on twitter
We’ve been over this before, but shot quality isn’t going away. Some people just can’t wrap their minds around the fact that a large amount of chance governs who wins and loses hockey games, and shot quality is a way in to unlocking this mystery. Also, shot quality is one of those things that clearly exists within a single game, and so the question becomes: If in every game I see, shot quality is a factor in who wins and who loses, how can it not be a factor over the course of a season?
The same people who think shot quality exists also want stats for board battles won, or passes completed, or the like – hyper-specific things that it would indeed be nice to know (I’ve forgotten who was doing work with passing here, but that looked promising), but they can’t let go of the fact that these things are contained within Fenwick/Corsi. If whatever a player is doing is not getting or preventing shots, it can’t be all that important.
Driving Play - The Blog with Three First Lines
There’s a guy doing a pretty interesting experiment by tracking basically every possession event for each player from puck battles won to passes completed and successful zone entries and what not for Montreal.
http://www.boucherscouting.com/
The results can be pretty interesting at times but I don’t think there is a huge amount of insight you find beyond something like tracking scoring chances in terms of level of play but it with the different types of play tracked it goes to show the many different ways to get there.
by Stephan Cooper on Oct 27, 2011 2:48 PM EDT up reply actions
It would probably be useful for apportioning credit/blame amongst players who almost always play together. WOWY
sometimes has such small samples for the without you portion that its not reliable.
Love this quote :
If whatever a player is doing is not getting or preventing shots, it can’t be all that important.
What I often find frustrating is when people argue in favor of the value of a particular tough player who has an awful Corsi. If all that hitting and intimidation is doing so much good why don’t they have a good Corsi? Either the rest of their game is so awful that the toughness can’t compensate for it or toughness just doesn’t make that much difference. Then they’ll say we need a player like Milan Lucic or some other 30 goal scorer who happens to be belligerent.
About shot quality, one of the main things is that a large percentage of scoring chances are a) going to be goals no matter who the shooter is or b) not going to be goals no matter who the shooter is. So I think the shooter only makes a difference on a minority of shots. Getting in place for a scoring chance is much more due to the player’s skill than converting the chance.
and even on the scoring chances that may or may not be a goal depending on the shot – whether it is a goal is largely luck.
I like to think of it like target shooting. The good shooters will have tightly grouped shots – the more accurate the shooter the tighter the grouping. So take that well-known picture of the net with the outline of the goalie and then pick a target where there is the most net to shoot at. Now group your shots around that target. The tighter the grouping the more will hit that area where there is open net. Each spot in the circle is equally likely – the shooter’s skill ends at the ability to keep the shot in that circle but he cannot control where in the circle it will go. Some spots in the circle are goals, others are saves, posts or wide. But its essentially random where the shot goes w/in the shooters grouping.
The tendency of higher and lower than average shooting percantages to regress to the mean implies that there isn’t even very much difference in the groupings between most shooters.
I also don’t think the speed of the shot matters that much w/in the ranges that are typical of NHL shooters. I think the vast majority of shots are blocked, not saved by reaction. On shots from a long distance I’m sure speed matters but on most shots the speed is high enough for the shot to go in if the goalie is not in the way.
The good shooters will have tightly grouped shots – the more accurate the shooter the tighter the grouping.
Will they? Shooters like Ovechkin and Kessel, who seem to have extraordinary shooting ability, are also some of the most frequent shooters in the league. It seems to me that accurate shooting simply isn’t sustainable.
Pension Plan Puppets
I hope YouTube comes down to film this.
Well, yeah, I agree - the best goal scorers may not necessarily have the most accurate shots but let me just assert
that there is some variability between players in how tight their shots group around their target.
My quote forgot about drawing penalties, which is important, but the tough guys you’re talking about likely take more penalties than they draw. But those guys aren’t going away – I do wish that people who liked them could just say ‘Hey, I like this guy, he’s not necessarily good, but I enjoy watching him and he doesn’t hurt the team that much.’
Driving Play - The Blog with Three First Lines
Gabe,
To what extent do you think SQ sucking as an aid is due to the scorekeeper issues discussed in Ryder’s recall? These things can be fixed by improving standards and helped by just adjusting for rink effects but to me it’s all about the sample size. When the difference between elite and bad is a few percentage points, it takes an absurd sample size to tease that out and scorekeeper effects or added variance from injuries and how often a team faces backups make it close to impossible.
That said, I’m not sure what a better alternative would be other than just save% for goalies and trying not to let shooting% have a big role for skaters, i.e. Corsi/Fenwick.
Driving Play - The Blog with Three First Lines
There’s lots that I agree with here. A couple of points.
1. I agree that there are better places for a team to be spending time than on save pct. This is especially true given the lack of disparity between the best goalie and the 31st best goalie.
2. I do think our shot quality models (including DIGR) are not explaining the majority of the variability in shot probability. Also, the DIGR methodology is an improvement over previous models simply because of the interaction between shot coordinates and shot type and the flexibility it provides.
3. There’s no doubt that as a community we would be better off if our data was more complete. We probably all can think of 8 or 10 things that we wish we knew that would make our metrics better. I hope we’re all open to new methods that might improve our explanatory/predictive power.
Finally, the idea that we cannot do better than coin tossing is absurd. It’s a simple case of conditional versus marginal probability.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
