Jody Shelley vs Ilya Kovalchuk – Who’s the better shooter?

Let’s go back to a familiar chart – Jody Shelley‘s observed shooting talent vs that of Ilya Kovalchuk:

Just a reminder: shooting talent is the amount by which a player exceeds the expected shooting percentage based on the locations he shoots from.  I've only included initial shots at even-strength in this analysis – no rebounds, no power-plays.

Looking at that chart, it seems apparent that Jody Shelley could be a better shooter than Ilya Kovalchuk.  The probability is quite low – less than 1% – but Kovalchuk has more than a 5% chance of being a net negative shooter, while Shelley has a similar probability of being a net positive shooter. 

How is that possible?  Anybody who's ever watched Kovalchuk play knows he's a slick offensive player who can pick a corner as well as anyone in the league, while Jody Shelley is the definition of 'hands of stone' – there might be several million people in the world who can shoot better than he can.  Those observations aren't wrong – they just aren't perfectly reflected in the limited number of shots taken by the two players.

So how do we resolve this?  First, consider every shooter who 200+ shots over the last five seasons and look at the mean and standard deviation of their shooting talent.  Next, we simulate a group of league-average shooters over the same number of shots.  This allows us to determine the talent and luck involved in shooting:

So we can now start with an underlying distribution for each player – with a standard deviation of 1.13% – and update it based on how the player did in the five post-lockout seasons.  We get the following distributions for Shelley's and Kovalchuk's talent:

Now we have a much different story.  When we assume an underlying shooting talent distribution, we get an entirely different story – in this case, the likelihood that Jody Shelley is a better shooter than Kovalchuk is about 1-in-10000.  And it's probably lower than that – I binned my results in a way that doesn't really give us precision beyond the fourth decimal place.  Regardless, it's clear that Kovalchuk must be a better shooter than Jody Shelley.

Here we can see Kovalchuk's observed performance versus his likely talent:

Kovalchuk's true talent level – as implied by five seasons worth of data – is approximately 10.2%, which is 3.4 standard deviations above the mean of our initial distribution.  There's a small amount of uncertainty on that z-score, so it's possible Kovalchuk is just (just!!) three standard deviations above the mean.  Either way, Kovalchuk is no worse than the second-most talented shooter in the league, and most likely the most-talented.  And there's no question that he's better than Jody Shelley.