Why Per-Game Metrics can't Evaluate the Draft
by Hawerchuk on Nov 20, 2009 9:00 AM EST
in Prospects
Email
PrintEarlier this week, I briefly discussed Malcolm Gladwell's assertion that there's no relationship between draft position and a quarterback's performance in the NFL. Gladwell cited a paper that showed that the per-play performance of quarterbacks drafted 100th or later was the same as that of quarterbacks drafted 1-10 overall.
Gladwell's claim is easily-debunked, but I wanted to give you an example of what kind of wrong results you get when you engage in these kinds of shenanigans. Let's look at NHL entry drafts from 1979-99 - the brown line shows the percentage of draft picks who eventually play at least one game in the NHL, while the blue line shows the average number of points scored per NHL game scored by those forwards and defensemen who played in the NHL:
If you look at the per-game performance, you would conclude that beyond the 20th pick or so, there's no difference between the drafted players. 250th picks who played in the NHL scored just as many points-per-game as 25th picks who played in the NHL. Hence, NHL general managers are terrible evaluators of talent outside of the obvious two dozen future stars. Time for me to drop a 'QED' and get my findings published in a reputable academic journal!
The only problem is that most late-round draft picks never play in the NHL. So the average played drafted 25th provides some value to a team at the NHL level, while the average player drafted 250th probably never gets invited to an NHL training camp. So one high-reward pick in the 9th round - say Henrik Zetterberg - makes up 90% of the per-game statistics and makes all of the guys who never played in the NHL look a lot better. (Or as they say of Bill Gates: when he walks into a room, everybody's average net worth goes way up, but the percentage other than him who billionaires did not...)
The correct conclusion is that general managers do a good job of evaluating draft-age talent. Sure, there's some selection bias in our chart - first-round draft picks will get many more chances to establish themselves - but over the course of 20 years, players get drafted in the order of how they'll ultimately perform at their peaks. A monkey at a dart board absolutely would not produce better results.
9 comments
|
0 recs |
Do you like this story?
Comments
Unfair competititon?
You against Malcolm Gladwell is like a debate between Barack Obama and Sarah Palin. Both look pretty good and will appeal to their supporters, but one will be paying attention to detail while the other will be spewing superficial sound-bites. You know who has my vote.
On another note, I’d be impressed if you could find a correlation between “draft position and a quarterback’s performance in the NHL”, barring some cross-sport freaks!
There is also some bias in the late round selections from the late 90s until around the lockout: a lot of teams picked European league veterans (particularily goalies) in the later rounds instead of an 18 year old kid playing 2nd line on his varsity squad in Boston. They bias things for comparing the 7th to 9th rounds vs. the 4th to 6th rounds, for example.
Hockey blogging can't get any flatter.
Gladwell has changed his tune somewhat and is now referencing Massey and Thaler’s well-known paper, The Losers Curse, “http://gladwell.typepad.com/gladwellcom/2009/11/more-on-quarterbacks.html” . His point is now that NFL GMs are not “rational decision makers” because they most often pick QBs without regard to surplus value. This is, of course, quite different from his original argument.
It’s important to note that Gladwell’s original QB argument was intended to support his main thesis that teachers should be hired from the general population of college graduates and education schools should be bypassed. Well, I’ve seen this in action, pre No Child Left Behind, California inner city schools experienced a teacher shortage and hired people with no teacher training on provisional contracts-I’m sure if Gladwell wanted to, he could find statistics on how this worked but I doubt they would support his point. (When No Child became law, districts were penalized for having “unqualified” teachers and qualification was defined to include teacher training certification so this doesn’t happen very much anymore). My own observation is that most of those hired this way (no certificate) were failures who didn’t complete the training that districts gave them or very driven to distraction by the kids. There were a few gems, but they were about as rare as a Henrik Zetterberg in the 9th round.
by Big Picture Guy on Nov 20, 2009 10:15 AM EST reply actions
Here’s what Berri had to say on his blog:
“this is because in the NFL…draft position is linked to playing time. And this link is independent of performance.”
I think this is demonstrably not true (#1 draft picks do ride the pine if they suck) but Gladwell still insists that it is even in this new post.
Berri’s condescension is evident: he says Pinker’s argument was supported by “stuff he found on the internet.” If its not published by a university press, as Berri is, its just “stuff” and not even worthy of a citation.
by Big Picture Guy on Nov 20, 2009 12:52 PM EST reply actions
Is there a way to combine this data? Since the percentage curve isn’t linear, and the PPG is more or less flat, it could be that some segments of the draft are overweighted. This is a far cry from saying that GMs do no better than a monkey with darts, but it might reveal some slight biases.
Great post.
I can’t believe this was even a debate.
Maybe, MAYBE if Gladwell said all players past X pick that eventually make it play at the same PPG rate or whatever as everyone past the 20th pick but that requires do many ifs that it hardly makes any good argument other than there are good steals sometimes later in the Draft.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
