The MIT sports analytics conference is always on my birthday, and it just seemed like a sad way to spend it when they invited me to appear on the hockey analytics panel. That, and I'm way too cheap to drop $2k on a weekend trip to Boston. But everybody I know seems to be emailing me articles about Brian Burke's appearance this year, so I almost wish I'd gone. Almost.
David Staples has a transcript of what Burke had to say, and I actually find it pretty funny:
Burke said NHL teams are looking for any edge they can get, but he hasn’t seen anything in the 30 or 40 papers his staff goes through each year. "There has not been a statistical breakthrough that I’ve seen in hockey."
What have you done for me lately? The first statistical breakthrough was the mere recording of on-ice events, starting in 1997. Teams were flying blind with respect to almost all of the rest of the teams in the league at that point. The second statistical breakthrough is Vic Ferrari's - the impact of line matching and faceoff assignments on statistics, the transient nature of shooting percentage, and the value of anyone recording scoring chances.
At any rate, having your staff read a bunch of papers on the plane is no substitute for integrating rigorous analysis into your scouting department. Lots of people read articles about weight loss every year...It's the ones who put themselves on a plan who actually lose the weight.
Burke attacks standard, traditional NHL plus/minus, saying it’s "horseshit." "Everybody is looking for these ‘Moneyball’ breakthroughs. … I have yet to see anything that has value in terms of an alternative way of evaluating players." "This whole ‘Moneyball’ thing aggravates me anyway. … Nobody has ever won a championship with Moneyball," Burke said.
Goals and assists are also "horseshit", but that's another story. Burke's comments highlight a prevalent problem in the NHL - the belief that hockey is different than baseball because in baseball there was a massive amount of free hitting talent nobodies floating around, just waiting for the taking by the one team that had a spreadsheet driving its decisions. The truth is that by the time the Oakland A's were challenging for the AL West, they had very few contributions from players of this ilk. (Just to jog your memory: Olmedo Saenz, Jeremy Giambi, Frank Menechino, Scott Hatteberg, Erubiel Durazo, Marco Scutaro and Bobby Kielty.) That is, there was no quick fix - 1998 was really the pinnacle for acquisition of this type of player: the A's had Matt Stairs, Rickey Henderson and Dave Magadan on the roster, and they accounted for roughly half the offense on a very bad team.
As for the notion that no one has won a championship with statistical analysis, Staples correctly notes: "Didn’t the Red Sox use a huge amount of stat analysis? Didn’t the Dallas Mavericks?" And let's not forget the New England Patriots. But it's disingenuous to claim that the team that hired Bill James did not win a championship with Moneyball.
Burke said hits vary from building to building. "You get the stat sheet and you wonder sometimes what they’re watching." Sometimes a "hit" is given for brushing a guy, but it gets the same credit as bashing a guy against the boards.
Analysts have found ways around these and many other similar issues. Of course, the presumption is that hits are actually an event that contributes to winning; nobody has even shown that to be true. Brian Burke may just as well be complaining that the NHL's attendance figures are wrong.
How about Milbury?
"There’s difficulty in hockey: who measures the scoring chance? Who measures the quality of the scoring chance?" Milbury said.
As the scoring chance project has shown, you don't need hall-of-fame goaltenders tracking chances for you. And I think most large data-gathering projects have had the same outcome - as long as you impose some general guidelines on your data collection, you end with something very usable - much like the flawed but exceedingly valuable NHL real-time stats tracking system.
And one last item:
Michael Schuckers, the only hockey researcher on the panel, said that teams can control number of shots they face, but not necessarily the quality of those shots. Faceoffs wins and losses are over-rated, Schuckers said. "You’ve got to win 100 face-offs to get a goal."
One marginal goal is worth roughly $500,000 in the NHL today, making a faceoff win worth $5000 on average and even more late in the game. The average NHL player makes $25,000 in marginal salary per game (above the NHL minimum.) If you have a center who wins three extra faceoffs (like, say, Scott Nichol or Yanic Perrault or David Steckel, among others), he pretty quickly makes his money back even if he has very little in the way of other talents.