In a piece that's all-but-lost to anything but the WayBackMachine, I contemplated the shootout just a few tens of games into the 2005-06 season. I was surprised to find that players who scored fewer than 14 goals per 82 games were just as likely to score in the shootout as players who scored more than 28 goals/82. Of course, all that says is that players with a general goal-scoring touch aren't necessarily the best scorers on breakaways.
But my friends over at Puck Prospectus wrote several articles on the shootout lately, including this one that refers to Alex Kotalik a shootout specialist (he has a 53% lifetime shooting percentage in the shootout), and another one on optimizing the order of your shooters. Clearly you want your best two shooters to go first and second. At any rate, this discussion made me wonder if we now had enough shootouts for a player's totals to have predictive value - in other words, have players taken enough post-OT penalty shots for us to say with some confidence what their true talent levels are?Let's first look at players who took at least five shootout shots in the inaugural 2005-06 season. That's 54 players. Let's then divide them into two groups - players who scored at a higher-than-average rate, and players who scored at a lower than average rate - and look at their shootout performance going forward:
So going forward, the high-scoring group were just as likely - 34% - to score as the group with the below-average shooting percentage. But maybe five shots isn't enough; let's look at players who had at least 10 shots in the first two seasons of the shootout and see how they did in the next two seasons combined:
This time we have 64 players in our sample, and again, the group of players who performed way better in Years 1 and 2 were indistinguishable from the group of below average shooters in Years 3 and 4. Ok, so perhaps I've still sliced that a bit too thin. Let's look at players at players who had at least 20 shots during the first three seasons of the shootout and run the same process (I selected 20 shots here because if I use 10, it still shows no difference between the two groups. This is true even if I compare just the top 1/4 of players to the bottom 1/4, players who - if we believe past performance has predictive value - should have a large difference in skill level.)
|05-06 to 07-08||2008-09|
So here we have just 34 players in total. The above average group regresses halfway to the mean, while the below-average group stays at the mean. But if that group shoots at the league average - 33% - that's not really a below-average group, is it? Indeed, the below-average group got just as many shots as the above-average one because once we've whittled our way down to 34 players, that's barely one shooter per team, which means these guys are their team's best options. Calling this group "below-average" also fails the sniff test - the 2nd- and 3rd-worst shooters in the group are Alexander Ovechkin (9/31) and Ilya Kovalchuk (6/28), who lack for breakaway skills in no obvious way.
The problem with assuming past shootout performance is predictive of future performance is that the sample size is too small. When coaches keep calling a certain player in the shootout it's presumably because he's shown breakaway abilities over the course of far more practice and game repetitions than the few shootouts we get to see. That's why the "below-average" groups both get to keep shooting even though they've done poorly in the past and also ultimately end up with a league-average shooting percentage. Their coaches are good enough to assess who has league-average talent.
The only deviation we appear to see from this is that the absolute best few shootout takers in the league appear to be slightly above average. Our top 14 players who shot 48% probably have a true talent of 41%. The top 7 players - Brad Richards, Jussi Jokinen, Ales Kotalik, Slava Kozlov, Viktor Kozlov, Saku Koivu and Erik Christensen - scored 54% of the time in their first three seasons but regressed to 45% in 2008-09, roughly the same regression to the mean as the entire group of 14. So a handful of very good shooters have 45% talent; the next ten have 37% talent; and the vast majority of the rest of the players in the league who get to continue in the shootout are right around the 33% league-wide average.
And if you add a 45% true talent shooter to an average shootout team, what do you get? A 54% shootout team. With an average of 10 shootouts per team over the least four seasons, that's a 0.4 point boost, or 0.2 wins. When you get that for free, it's nothing to sneeze at - it's double what I estimate the value of a good fighter is - but it's still on the order of what you get from having the equipment guys make sure nobody's playing with an illegal stick.