The League of Extraordinary Statisticians: The Awards

The League of Extraordinary Statisticians (LOES) is a weekly forum bringing together top analytical minds in the hockey world to answer a variety of questions that straddle the line between stats analysis and something you might hear floating around section 304. They have agreed to answer these questions in a few paragraphs or less, and with minimal formulae. Because this is a forum, we’d encourage you to use the comments section to answer the questions yourselves, or to discuss or debate the answers given.

The LOES is not meant to represent the entire of the hockey stats community. There are a number of people that either were too busy or too difficult to contact for the purposes of the forum.

The LOES is fresh from the All-Star Break, hoping that the time-off won’t affect their performance over the final stretch of the season. The good news? We will have a couple of additional statisticians participating in the latter half, and I’d just like to say it has been awesome to work with these guys to this point. The bad news? My grandpa. Anyway…

You might remember that awhile ago we had the LOES look at “lifetime Trophies” and who they would reward for their NHL careers. Well this one is going to be a bit different; one of our readers, ‘red army line‘, did a Fan Post in mid-January asking an interesting question: should NHL awards be given based on “talent” or “performance?” The idea is that a “performance” (of, say, 40 goals and 72 points) does not tell us everything about that player, and metrics like Quality of Competition or Corsi Rel could allow us to get a better idea of that player’s talent within than particular season. Presumably, this would temper our feelings about a defenseman that scored a lot of points but was a disaster in his/her own zone, or bring us to place more value on a forward that shuts down the opposition’s toughest players in addition to putting up respectable point totals.

More after the jump…

The post was very interesting to me because it was an honest attempt to look at NHL awards differently.  I figured that to have the LOES review the post and respond to it would be a fun activity, and maybe generate some further ideas on how to alter existing awards.  Our LOES's responses to awards were interesting, alright.

This week's question: Based on 'red army line's' post, should NHL awards be given for "talent," "performance," or a combination of the two?

I remember hanging out with this really smart guy named Brendan back in University, who earned a perfect 4.0 GPA in first year computer science. When someone once said to him “I could have done that,” Brendan responded that “the odds that you could have done it are equal to or less than 100%, whereas the odds that I can do it are exactly 100%.” (P.S. Brendan didn’t have a lot of friends). The point is that while other students had the talent to match Brendan’s achievement, only he who actually
accomplished the feat truly deserves the recognition.

What’s true of teenagers who drink Coke with breakfast and don’t comb their hair is also true of elite NHL athletes. While it’s unfortunate that some players may have had the talent to perform at a higher level but simply lacked the opportunity, it’s important to remember that it’s still a theoretical argument. What one player actually does outweighs what another player theoretically could have.

– Rob Vollman, Hockey Prospectus

The voters don’t really know, and it’s easy to be confused. The Hart goes to the most valuable player to his team, the Lindsay to the most outstanding player, the Norris to the defender with the greatest all-around ability, the Calder goes to the most proficient rookie, Vezina goes to the best goaltender. So there you go – you can choose from, and muddle between, the best, most outstanding, most valuable, most proficient or the most ability. For all of these trophies I like to interpret all these definitions of “best” as “made the biggest contribution to team success”. This is the Hart Trophy definition and is the easiest and most logical definition with which to work.

Let’s further clarify the issue by observing that Contribution = Opportunity x Effort x (Skill +/- Luck). You can’t have impact if you don’t have the opportunity to perform – you have to play. Very few think of Bobby Orr as the greatest player of all time, mainly because his career was too short. Sidney Crosby won’t (and shouldn’t) be MVP if his headaches don’t subside soon. You also can’t have impact without effort. So it is clear that “skill” is, by itself, meaningless. The real question is “what to do with luck?” The good news is that the ‘error term’ in the equation (“+/- Luck”) is dampened out by playing time. This is the second big reason to insist on a measure of greatness that is heavily dependent on playing time.

But, at the end of the day, we can’t reasonably separate luck from skill as the latter is unobservably intrinsic. And perhaps we should not. After the 2007-08 season I said this about Tim Thomas: “There is a rich history in the NHL of late-blooming goalies. But expect his performance to regress next season.” In 2009 he won the Vezina (oops) and I said, “I still expect regression” (a really easy prediction to make). Last season he regressed, just enough to lose considerable playing time to Tuukka Rask. But this season he is back as the NHL’s most valuable goaltender – with what appears to be a save percentage poised for more regression…

It has been said that “luck is when preparation meets opportunity”. Talented players know what to do with opportunity. Let’s give them credit for their performance rather than what we think they are.

– Alan Ryder, Hockey Analytics

I must confess that I read that fan post and I am not exactly sure what the distinction he is trying to make is exactly. as a result, I cannot say for certain if I agree or disagree with his point.

My best interpretation is that he is drawing a distinction between “performance” i.e. who played the best that season and “talent” which I interpret as if the season were to be repeated an infinite number of times, we would expect an infinite number of results – now who would have the best season the highest percentage of those times. If that is his distinction, awards should always go to performance. For example the defenceman who plays the best in the season in question should always win the Norris Trophy that season. It makes no difference to me how likely the season was, it is the season that actually happened.

That may mean that we get award winners who do not hold up well when it is looked at in the context of their entire career. For example, in 2002 Jose Theodore won the Hart Trophy. He was the best player that season. The fact that it wasn’t a likely occurrence or that he would never repeat that season doesn’t matter. The fact is he had that season.

Of course if that isn’t the distinction he is making (and I am not entirely sure that it is), then I wrote a few words that miss the point.

– Greg Ballentine, The Puck Stops Here at Kukla’s Korner

Performance, although I can see how the line can quickly get blurred once you add caveats. Awards should be based on performance…but based on “meaningful stats”. Yet once you starting talking about the nitty-gritty of what “meaningful stats” are, does that mean that you need to adjust current performance for luck (for instance)? Because by doing this, you’re arguably entering some gray area between performance and talent already. In practice, the simple answer is performance, but you’d like the voters to be at least somewhat educated about which stats are pretty solid to base their decisions on and which ones are basically crap.

– Timo Seppa, Hockey Prospectus

Call me a curmudgeon, but I don’t really care about NHL awards. At their best, picks can be based on performance – though many times performance isn’t even evaluated correctly by the voters. I can’t
even imagine how they’d evaluate talent!

– Gabe Desjardins, behindthenet.ca and, of course, Behind the Net

While it was a very good fan post, I think the answer is obviously “a combination of the two”. You have to have talent in order to perform, and you have to perform in order to win awards. Talent alone doesn’t translate into results, but performance alone isn’t enough. As alluded to, there are plenty of other factors that need to be taken into account besides a player’s “boxcar” stats, but you need to factor in luck at some stage. And situations.

But really, when talking about NHL Awards, it’s a pointless endeavor. Winners will be selected based on points, plus/minus, and name recognition.

– Geoff Detweiler, Broad Street Hockey

I’m not a fan of awards, but if the league is going to continue the tradition, it’s time to make some changes. As long as the PHWA lords over the vote, the awards are going to be awards on neither talent nor performance. They’re going to be awarded on narratives and storylines. Some PHWA writers do a very good job and take the time to consider their votes. Others just follow the narratives they establish throughout the season and cast whatever suits the story. Essentially, the awards somehow avoid both talent and performance. It’s time overhaul or at least expand the voting pool.

Decide who the awards are for and include them in the process. If the awards are for the players, bring them in on all trophies. Let each NHLPA player representative cast a vote. If the awards are for the fans, bring them in on it through an internet vote. If the awards are for the league, allow each GM to cast a vote on each trophy. If the awards are for all of the above, give 20% of the vote to the PHWA, the NHLPA, the Fans and the GMs. Give the remaining 20% of the vote to previous award winners.

The league is never going to award anything based on true talent or actual performance, so make the season-ending awards about the game as a whole. Break the PHWA monopoly and bring some legitimacy to the awards. Reach out to all of the representative groups associated with the game and in the end the process and ceremonies will be a better fan and league experience.

– Derek Zona, The Copper & Blue

A bit of frustration, eh? In all seriousness, even the casual fan has experienced a certain amount of frustration over players they feel were slighted for x award. But the statisticians frustration is compounded by having any number of quantifiable arguments to support their claim, only to see well-deserving players continually overlooked. It has only been in the last couple of years that the Cy Young began to go to pitchers with low-win totals and solid peripherals; how many years before a Dennis Seidenberg or Mike Weaver gets the Norris (interestingly, the high-flying 1980s produced our last mainly-defensive Norris winner, Rod Langway)?

The more I read these responses, the more I regretted not asking a further question myself: what about doing away with the awards? A draconian response, sure, but should an arbitrary award become a focal point for deciding on future Hall-of-Famers? Especially considering coming in second in voting for any of these awards essentially means nothing in HOF decisions? What would you do to change the NHL awards? By the way, I love Derek’s suggestion of giving former winners a part of the vote; who wouldn’t love Langway telling Caps fans why he didn’t vote for Mike Green? Oh right…Caps fans.

P.S. Tom Tango had a more philosophical response to 'red army line's' post back when it first came out; it's worth a bit of your time.