clock menu more-arrow no yes mobile

Filed under:

Further to The Big Mistake

Where human intuition and the eye-test could be failing you.

Fred Greenslade-USA TODAY Sports

I like to read, a lot. With years of solid research and opinion articles available, the advance hockey statistics blogosphere has consistently fed my addiction. Often, I will link to these articles in my own. I also like to expand further and test the things I may agree or disagree with, maybe even more than I develop my own ideas.

One of my favourite articles comes from Tyler Dellow, The Big Mistake. He opens up with this doozie of a quote:

If someone asked me what I think the biggest failing of the eyeball test is, I’d respond that it’s the emphasis on the big mistake. There are gigabytes of information contained in a hockey game. So much information that I think it’s difficult for anyone to take it in and organize it rationally. The way that our brains deal with that is by focusing on the big mistake.

I would expand this hypothesis even further; the biggest failing to the eyeball test is human nature and its propensity to look for direct causality and responsibility. The tendency to look for a target to blame or extol is not limited to defensive contributions.

Even with the short history of Winnipeg Jets 2.0, there are multiple examples of this phenomena.

The recent announcement of Alexander Burmistrov's decision to stay in the KHL for another year brought up some of these topics. Underlying statistics have often smiled on the young Russian centre. The numbers indicated that Burmistrov helped improve his linemates as well as the Jets chances of winning, despite his flaws and lack of dynamic offense. (For this article, we'll ignore that Burmistrov's production issues were exclusively when placed with Jets non-scoring players and discuss these ideas more conceptually).

We can see this for example by looking at Evander Kane's 2011-14 numbers with different centremen:

Player TOI Corsi% WOWY rel G/60 A/60 P/60
Olli Jokinen 773.2 50.9% -0.30% 0.47 0.93 1.40
Alexander Burmistrov 757.4 53.2% +2.80% 1.43 1.03 2.46
Mark Scheifele 379.2 51.4% +0.30% 0.79 1.11 1.90

WOWY rel is the delta between Kane's Corsi with and without the player.

Why is that? Both Jokinen and Scheifele are considered superior scorers and have scored at better rates than Burmistrov; Scheifele was even a better scorer in junior. From a "watch the game" perspective, Scheifele also looked to have awe-inspiring chemistry with Kane, while consistently connecting on plays in the offensive zone.

Maybe, just maybe, a player can affect their team's chance to score without directly contributing to goal scoring.

We can see this here with Evander Kane's shot rates and goal scoring rates compared to his Corsi%:


In the graph we see that Evander Kane scored more when he shot more, and shot more when he was in the other team's zone more.

While this does not prove causation at any extent, it does seem intuitive. A player who helps their team play in the offensive zone more often gives their teammates more time to create chances, and therefore indirectly improve scoring.

I've heard from more than one source that the Jets use Nielson numbers in player evaluations and scouting. For those that do not know, a Nielson number is a scoring chance plus/minus where only the players who directly contributed to the play gain a plus or a minus. Nielson numbers are a great coaching tool. You can use Nielson numbers to help target where a particular player may have an issue in defensive zone coverage and the like; however, the numbers fall for the same big mistake as the eye test with missing indirect consequences due to choices.

The recent birth of neutral zone microstatistics -such as zone entries and exits- have enlightened us to where some of these conflicts arise between eye-test driven perceptions and on-ice shot metrics. The general practice in the NHL as of late has been that chip-out and dump-in plays, often called the "safe" play. Why? Because they diminish the chance of a giveaway which can lead to a goal against, ie: the big mistake. However, recent research has shown how the these plays are most often voluntary losses of possession, and can indirectly lead to more goals against and less goals for over the long run, ceteris paribus.

A defensive zone example is the perceptional difference between Mark Stuart and Dustin Byfuglien. Stuart is often praised for being safe, even though in the long run he is a greater danger. His inability to play the puck outweighs his positive on-ice attributes. Meanwhile, Byfuglien is chastised for his turnovers even though in the long run he helps more than he hurts. Byfuglien, as a puck-mover, has the puck more often, leading to more chances for large-event and memorable error, but overall improving the team.

With Burmistrov, the most common critiques were that he was not playing a "North/South" game and held on the puck too much. People complained that because of this, Burmistrov was losing the puck too much when entering the zone, even though this turned out to be untrue. In the end, the Jets' percentage of time in their opponents zone in 2011-12 and 2012-13 was 3.4 and 2.8 greater with Burmistrov on the ice than without. Those two boosts were 5th and 2nd highest on the team.


Looking exclusively at direct causality to events is extremely flawed in a sport where indirect contributions occur all the time: zone entries, zone exits, denial of zone entries/exits, board battles, blocking a passing option, etc. These indirect contributions may overpower a player's direct contributions in goal scoring creation or suppression. Fortunately, shot-metrics cover overall contributions, regardless of being direct or indirect.

Sometimes you will hear a counter to a stats based argument with saying that hockey is different than baseball, and that "moneyball" can't work for hockey. Moneyball was simply about market inefficiencies; it showed how something a person intuitively believes to be the largest variable in producing wins may not actually be. Moneyball was about adjusting how particular things are valued.

It is ironic that a common argument against statistics is that it doesn't represent everything, when the biggest fault to the eye-test is that most non-direct contributions are either missed or forgotten. What remains is the big mistake.