Retro NHL and Anger at Corsi
by Hawerchuk on Nov 25, 2010 8:00 AM EST
Email
PrintI was going to write something funny about turkeys today, but I couldn't think of anything. Instead, I'm going to talk about the irrational anger directed at shot differential. Even though NHL insiders as old-school as Harry Sinden used it as a metric to evaluate their teams, the current spate of rejectionists seem to think that shot differential is some bastardization of the original intent of the framers of hockey's constitution.
Let's walk back a few months to Derek Zona's post about the demise of the Colorado Avalanche in last year's playoffs. After a 69-point season in 2008-09, nobody thought Colorado had a chance to make the playoffs last year. But Colorado took a brilliant flyer on Craig Anderson, who'd put up great numbers at every stop during his career, and it was about to pay off. When the Avs opened the 2009-10 season 9-1-3, they were almost guaranteed to outperform expectations and make the playoffs, regardless of their underlying possession, scoring chances and shot differential numbers.
And Colorado's underlying numbers last season were horrible. But they won a lot of games. If you were a fan who didn't care about statistics, would you really care what they said about your team? Wouldn't you just enjoy watching your team win? We know the San Francisco Giants got about as lucky as possible this past season, but the Bay Area's fairweather fans were able to enjoy the victory just fine whether their team performed over its head or not. Apparently that's not how things work in Colorado:
"I just don’t quite get the glee from all corners for pointing this out. Shouldn’t the story be how the Avs are still absolutely blasting preseason expectations?"
And here I thought I was spending a tremendous amount of time discussing the Avs exceeding performance expectations. Should we not seek deeper explanations? Must we simply assume that Colorado is a truly great team? Our critic goes on, addressing his complaints to "Gabe and the Corsiatti":
"As it relates to this discussion, about the ‘10 Avs being lucky, I’ll summarize your logic I took issue with thusly:
Team Corsi ratings correlate to team record to some degree.
The ’10 Avalanche team Corsi ratings were poor.
The ’10 Avalanche had a much better record than Corsi predicted.
Therefore, the ’10 Avalanche were lucky.
All of the premises are true. The issue with that conclusion is that it assumes that the only reason the prediction of Corsi could be wrong is luck. It’s a classic false dichotomy. I pointed out that such a conclusion requires the link between Corsi and record to not just be a correlation but a causal one (which you confirmed you do believe, despite a lack of any support for this claim). But since the only link between records and Corsi is some level of correlation, that’s by definition an invalid conclusion, unless you’ve got a lot more data and reasoning somewhere you’ve not deigned to provide. Based on what I’ve seen, all you can say is that Corsi’s prediction of the ’10 Avs was wrong.
You also keep claiming the burden of proof is on me somehow to prove the ‘10 Avs weren’t lucky. I don’t know that it wasn’t luck, much the same as you don’t know it was. It very well could have been luck. I have pointed out that your model is limited, and that perhaps considering other factors might improve it."
(That's the most coherent and least personally-insulting part of what the Corsi critic had to say, and I've left out the parts where he says "I feel like beating people over the head with correlation vs. causation lectures.)
Anyways, I think I see the issue here. This is not a "classic false dichotomy" wherein only two alternatives are considered, irrespective of other options (though if it was, I could see why it might bother some people.) The thinking behind Corsi is much more nuanced than that and despite what its detractors claim, it's not some kind of religion. Here's the basic idea:
First of all, if you've never visited Objective NHL, you're really missing out. Almost everything that goes into team-level predictive analysis in this so-called "corsi" model is derived there in gory detail. If anybody claims that something is unproven or that there are no numbers behind it, they can go there and almost certainly be overwhelmed by the numbers behind every assumption.
At any rate, we want to find what talent factors drive a team's winning percentage. Goals for and goals against drive outcomes, and they are very highly driven by shooting percentage, which itself is not a sustainable talent. There are plenty of ways to figure this out - comparing first-half team shooting percentage to second-half; or even games to odd ones; or even shots to odd shots. None of them show a persistent relationship.
What does show a persistent relationship is shot differential, in particular shot differential with the score tied at even-strength. The best predictive performance comes from goals, saves and missed shots taken together - blocked shots are driven to a great extent by a team's ability to block shots, so they're not as useful. This is what's commonly-referred to as "Fenwick", while Corsi (usually) includes blocked shots.
While Fenwick is the single-best predictor we have of team performance, it's not the only one. For one thing, team records over just 82 games are a poor sample of a team's ability, and much of a team's record is actually just luck. Don't believe that? Commenter Dan Lortimer has a great suggestion for you: go to NHL.com and watch the replays for a few hundred goals - count how many came off lucky bounces, and how many were skill plays. When you think about it that way, I think you know that luck is going to play a big role.
Together, Fenwick/Corsi and Luck account for around 3/4 of team winning percentage. What's the remainder? Goaltending talent - which Tom Awad estimates at about 5% - and special teams, along with a very small sliver that's due to shooting talent and the oft-mentioned "shot quality." So I don't think there's a false dichotomy here - there are five factors in this model, all of which are given credence in proportion to their predictive power.
***
So let's move on to this question of correlation vs causation. Yes, if you're sitting in your freshman philosophy class, you might argue that no such thing as "causation" can possibly exist because it requires that one particular thing causes another thing to be true. But in the real world, people are constantly running experiments to determine the extent to which one factor causes another. Does smoking cause lung cancer? Not 100% of the time, and not on a predictable timeline. Is smoking merely correlated with lung cancer, but not causative? You know, even after all these many years, we can't be certain in a philosophical sense that there's a causal link between the two. But does anybody honestly doubt that smoking causes lung cancer?
Now Corsi is a proxy for something else - puck possession, territorial control, scoring chance differential, all of which are almost identical metrics - and does anyone really doubt that having the puck more than the other team does not lead to winning? Yes, there are exceptions - faced with a "hot" opposing goalie or shooter, it's possible to lose a game, but "hot" streaks are transient - not skill-driven - and it would be incredibly bad luck (not skill) to run into them for an entire season. Goaltending and shooting talent have been studied by many different people in many different ways and have been found to drive only a small slice of the results - 1 or 2 wins per season.
The caution that "correlation does not imply causation" is a check on your assumptions, particularly the assumption that you've constructed equal groups in your experiment. That's why people compare individual teams by halves of the season or even and odd games or shots. At any rate, I haven't seen anyone dispute the experiment design, only the conclusions drawn from it. Well, every day people draw conclusions in just that way that Corsi was concluded to be the biggest talent-based driver of winning percentage. You can sit around paralyzed, debating the notion of absolute certainty, or you can say "You know, if the other team never gets a shot on goal, you'll win. Perhaps outshooting your opponents causes you to win more often that not. The data indicates that this is true, so a team talent model should include shot differential."
This shouldn't be a controversial conclusion. Is there a hockey coach in the NHL who disputes it? Is there a coach who revels in getting outchanced? Is there a coach who accepts that his forwards get penned in in their own zone and has a strategy that revolves around his goaltender bailing him out?
I know it's easy to *attack me* but if you're so convinced that we can never know anything, then you should be prepared to attack NHL-wide coaching strategies, and you should have some evidence to back up your claims. It's a total cop-out to say "I don't know but because your model's not perfect, you also don't know, so you should stop making conclusions and just shut up." We know full well that sheer luck can result in bad teams posting good records and vice-versa - without it having any impact on their true talent. If you really dispute the notions of talent and predictive modeling, then you should be out there fighting against more than just my analysis of the Colorado Avalanche.
133 comments
|
6 recs |
Do you like this story?
Comments
This is actually a great defense of that little thing some people call “the scientific method”. Nicely done. Happy Thanksgiving!
Of course they can’t, Gabe. How else can a player give 110% if science was allowed?
Devils in my heart! Devils in my mind! Devils in my eyes! Devils until I die!
In Lou We Trust - The New Jersey Devils SBN Blog
by John Fischer on Nov 25, 2010 8:30 PM EST up reply actions
Yeah but you're not taking into account 'stepping up'.
They may not be able to give more than 110% but there is no scientific reason why they can’t ‘step up’. This is why all explanations are just excuses for losers.
Does smoking cause lung cancer? Not 100% of the time, and not on a predictable timeline. Is smoking merely correlated with lung cancer, but not causative? You know, even after all these many years, we can’t be certain in a philosophical sense that there’s a causal link between the two. But does anybody honestly doubt that smoking causes lung cancer?
Moreover, we have a pretty solid mechanism to describe how this could (and frequently does) happen. So it doesn’t happen 100% of the time. All that means is that there are other external factors (better defence mechanisms against cancerous cells, for example) that could explain the variation.
And that’s the thing in the end. If you have a good model, which can explain both the main effect and as much of the variations as possible, you’ve got a workable theory, at least until it’s replaced by a better one down the road.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
Yes, but if enough of the elements of the model have been validated experimentally, anyone who says otherwise can pretty much suck an egg unless they can conceptualize and validate something better. ;)
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
I’m still stunned that someone—freshman philosopher or not—could say that having the puck more often than your opponent can’t be considered a cause of goals (if one among many). It’s mind boggling that someone would watch a hockey game and not come away with the conclusion that players fight for the puck almost incessantly.
If they wanted to argue with you, they would have to argue that corsi (or fenwick) can’t actually be used as a proxy for puck possession. But to suggest that this is a question of correlation vs causation…well, that’s kooky.
The best predictive performance comes from goals, saves and missed shots taken together – blocked shots are driven to a great extent by a team’s ability to block shots, so they’re not as useful.
I never understood that. If the shot isn’t blocked, won’t it result in a shot, or at least a missed shot, which would count?
Consider two teams, Team A and Team B, who are absolutely identical except Team B is awesome at blocking.
Let’s say against Team B you got 25 shots, 10 miss and 20 blocks. If you played the exact same game against Team A, fewer shots would be blocked, either leading to a shot, or a miss – let’s say 35 shots, 15 miss and 5 blocks.
So against Team B you had 35 Corsi, and Team A 50 Corsi – but you played the exact same game.
See: http://vhockey.blogspot.com/2010/05/blocked-shots-luck-or-skill.html
If you take G+S+M, you get the best predictive value. S+G and S+G+M+B have equivalent predictive value, and it’s slightly lower.
I’ve read that, and I accept it as truth, I just don’t understand why it is so. (because a failed B is an G, S or M, by definition).
by Rob Vollman on Nov 25, 2010 11:18 PM EST up reply actions
Is there a coach who accepts that his forwards get penned in in their own zone and has a strategy that revolves around his goaltender bailing him out?
The first thing that springs to mind is Team Switzerland or Austria when they play Team Canada.
I mean, they don’t “accept” getting penned in, but their game revolves around their goalie playing incredibly well and then maybe getting a lucky goal at some point.
Or back in the day with Dominik Hasek, the strategy (to an extent) was “go ahead, have the puck, blast away, we’ll wait until you over-commit and then make you pay.” Even a great team like the 70s Flyers with Parent in nets let the other team have the puck and take shots.
I’m not trying to defeat your argument, and I know I’m stretching, I just want you to say more on this topic. With a strong goalie and a weak team, it wouldn’t necessarily be unusual to worry a bit less about puck possession.
Switzerland isn’t trying to win though. They’re trying to tie, or really keep the game within one. Totally different strategy.
And yes, Hasek was probably just that good. So once in a generation, you have the opportunity to play a different strategy.
But don’t the statistics show that most teams seem use this strategy all the time? Few teams with a lead actually out shoot their opponents.
----------------------------------------------------------------------------------
HockeyAnalysis.com - Taking a Deeper Look at the World of Hockey
by HockeyAnalysis on Nov 25, 2010 3:29 PM EST up reply actions
What do you mean? Hawerchuk said Switzerland was so clearly outmatched, they basically were forced to play an unconventional style. With Hasek in net, the Sabres could afford to give up more puck possession and thrive on higher percentage chances off the rush. But most of the time, it’s puck possession, or at least has been especially since the lockout.
by red army line on Nov 25, 2010 3:51 PM EST up reply actions
The skill differential between Canada and Switzerland is much greater than the skill differential between any two NHL teams — to the point where Canada’s weakest player is stronger than Switzerland’s strongest (ie. Streit is good, but look at Canada’s top 6 D…) Such a large difference of skill level naturally alters strategy in a way you don’t usually see in the NHL. For example, Canada wouldn’t particularly need to match lines, when it can roll four NHL-first-line caliber units.
The initial claim was that no coach would want his team dominated and penned down in his own zone, and yet teams with the lead appear to be doing just that. Now they aren’t dominated in the sense of Canada vs Switzerland, but they do allow relatively more shots than they take and certainly more than they generally would at even strength. In essence they seem to be ‘holding on to the lead’ just as Switzerland seems to be ‘holding on to the tie’.
----------------------------------------------------------------------------------
HockeyAnalysis.com - Taking a Deeper Look at the World of Hockey
by HockeyAnalysis on Nov 25, 2010 4:21 PM EST up reply actions
So by now, your trolling act is wearing pretty thin.
Corsi EV score tied is what has predictive value – that’s what I wrote above. And you come in and say “But what about score effects?” as though I didn’t write it.
I get it. You disagree with me no matter what I have to say. Point made. Nobody cares because you aren’t offering any substantive points. Start writing all of those articles you promised you would.
I am commenting on one sentence that another commented pointed out. I am not disagreeing with everything you said.
Is there a coach who accepts that his forwards get penned in in their own zone and has a strategy that revolves around his goaltender bailing him out?
The point I am making is that under certain scenarios (i.e. when a team has a lead) it appears that a number of NHL coaches employ this strategy. I am not sure it is a smart strategy, but it seems to be a strategy nonetheless.
BTW, Corsi Ev when down has predictive value too according to the same theory you use for Corsi Ev tied. It is slightly lower than Corsi Ev tied, but not too far behind. Corsi Ev when up is predictive as well but much less so.
(BTW, don’t take this statement as me accepting Corsi, just an observation. I still wouldn’t trust it, especially in individual player evaluation.)
----------------------------------------------------------------------------------
HockeyAnalysis.com - Taking a Deeper Look at the World of Hockey
by HockeyAnalysis on Nov 25, 2010 8:31 PM EST up reply actions
David, you gotta read what I wrote. Ev score tied. Score effects are something different
by Hawerchuk on Nov 25, 2010 4:12 PM EST via mobile up reply actions
But what causes scoring effects? Is it a tactic by the team in the lead to allow shots but limit quality shots and odd man rushes? If that is the case, how is that different than what Switzerland might do against Canada?
----------------------------------------------------------------------------------
HockeyAnalysis.com - Taking a Deeper Look at the World of Hockey
by HockeyAnalysis on Nov 25, 2010 4:30 PM EST up reply actions
Can we talk about professional hockey? In junior or the ECHL or the Swiss hockey league, different strategies apply. When an NHL team is up a goal, they go into a defensive shell. But Switzerland goes into a defensive shell with the game tied or even when they’re down a goal. NHL teams are trying to win. They’re not looking for a moral victory by only getting beat by two goals in Chicago.
Remove head from rectum
Why can’t people accept the Avs as an outlier in 2010. For reasons extremely difficult to statistically evaluate.
I would guess it is because they perceive that being an outlier means that you are stating that they didn’t ‘deserve’ to make the playoffs.
It’s not an attack on the team or by extension the fans of that team and it doesn’t belittle their achievement in making the playoffs – they just didn’t do it in a normal or expected manner, and statistics show how it was irregular.
I would guess it is because they perceive that being an outlier means that you are stating that they didn’t ‘deserve’ to make the playoffs.
BIngo, that’s exactly it.
If any teams didn’t deserve to make the play-offs, it’s the Bettman teams – the teams that only made it with baloney OT/shoot-out points (ie. Habs over the Rangers).
by Rob Vollman on Nov 25, 2010 11:21 PM EST up reply actions
This is good stuff, but it seems like the people that are criticising your approach aren’t really going to accept your views and most everyone else probably already understands the scientific method.
Not to stoke the Avs-centric stats analysis going on here, what are your views of the Avs this year so far? To my eye they seem to have improved overall – but their defense is still brutal at times.
Nitpicks: To be fair, your Philosophy 101 class will tell you that it’s impossible to say with certainty that causation exists, not that causation doesn’t exist. And I think it’s actually pretty well established that while smoking might not directly cause cancer, many of the chemicals found in tobacco do cause cancer.
But really, I don’t have a big problem with Fenwick/Corsi. I think it’s occasionally referenced to state more than it does, but that’s an issue with the interpretation rather than the model. My issue is more with “luck.” Yea, sometimes lucky/unlucky things happen, but generally they’re at the tail end of actual work. If a team reliably goes deep into the zone, they’ll likely have more “luck” than a team that takes a bunch of shots from the periphery.
I’ll admit, it’s mostly semantics, but when I hear statements like:
Together, Fenwick/Corsi and Luck account for around 3/4 of team winning percentage. What’s the remainder? Goaltending talent – which Tom Awad estimates at about 5% – and special teams, along with a very small sliver that’s due to shooting talent and the oft-mentioned “shot quality.”
it bugs me that luck is used as the catch-all. I mean, realistically, given your list, it seems that coaching/strategy, preparedness and team play all fall under luck. It assumes that the better team should win most of the time. But we only have a mathematical feel for teams as an average and that average is weighted as a byproduct of the division and conference the team is in. So deciding which is the better team, especially on any given night, is up to some debate.
I don’t think there’s anything particularly wrong with the model – my issues are with what the model doesn’t treat as unique and important. I realize that dealing with divisional weightings and team strategies and team play is more or less impossible given the sample space that is available and because of that the current model really is the best predictor we have, and there’s not a lot of room for improvement.
Were the Avs outliers given the model? Yes. Were they lucky? Sure. Every team plays somewhat above or below their theoretical maximum. But I’d personally hesitate to say that they were/are a bad team or that they were just plain lucky. There’s some wiggle-room on the luck front due to specificity and sample space and I don’t think anyone can say anything stronger than “they were an outlier” with any measure of certainty.
I should probably add that as much as people hate things like “clutch” and really anything else to do with psychology or “intangibiles” (likely because they don’t really fit into a mathematical model as predictable behaviour), they certainly exist. And in real, data-based science! There are decades worth of physiological and neurological articles which establish pretty concretely that they are real things. For example, stress affects memory storage and recall – but different people/rats/etc are affected by different levels of stress differently. What might be a perfect stress level for one player to thrive might be way too much stress or not enough for one or both of their linemates. And then there’s the issue of self-imposed stress and whether or not any of the players on the ice are equally stressed at any point in time.
And then you have mirroring effects such that seeing someone eat a piece of cake lights up a lot of the same places in your brain as if you were the one eating the cake yourself. Or that the amygdala can trigger the release of neurotransmitters that can amp you up or relax you, and it isn’t at all consciously controlled. Or that your neurological zombie — think breathing — does its job much better than you do when you consciously try to do it. It’s not only with breathing, but also with motor movement and reflex. When Gretzky talks about getting into the zone, he’s (according to recent neuroscientific research) likely talking about giving himself entirely over to the zombie Gretzky who’s just really good at hockey — taking consciousness out of the picture. Yes, you train your zombie with everything that you do, but sports psychology can enable you to give over to your zombie more quickly and readily.
It’s actually a fascinating field. Not at all related to predictability, and that’s why some players are more consistent than others, but there’s an amazing amount of real, numbered data that show that intangibles not only exist, but also that they have a direct effect which is most visible at high levels such as professional sports.
It’s why I don’t like the word “luck” as a general rule.
by SNNEnder on Nov 25, 2010 7:31 PM EST up reply actions 1 recs
Resistance to reality
SNNEnder your condition is very common.
This site gives many evolutionary reasons for it!
One of the main ones is the need for Control! and security)
(See Why we see patterns in Random Events)
http://www.problemgambling.ca/EN/ResourcesForProfessionals/Pages/WhyWeSeePatternsinSequencesofRandomEvents.aspx
You may not like it or accept it but not liking luck is akin to not liking ‘gravity’ —
or accepting its reality?
randomness (which is a better term than luck)
is the driving force in most sports performances! 65% in the NHL
All the factors in your post if they have any effect – we must ask what precisely to they influence.Do they improve ‘work effort’ for example? As fans and sports talk radio shows talk ad nauseum about how my team isn’t working had enough. But there is no meaningful correlation between hard work and success. Or any of the other intangibles in your list.
Here’s an experiment
Given that we know that NHL is influenced to a great degree by luck.
Any repeatable pattern over seasons would be impressive and a sign that
the team in question has a certain skill that’s driving performance..hmmm…now which team has been the most successful over the last 15 years
Detroit – ave point total 110. 1.34 per game 68% w% in a 55% league
This is a team to examine to discover ‘clues’ to what skill drives win %
Surprise! there shot diff. (not even Corsi) also is approx + 7
Suprise #2! there current coach speaks about ‘puck possession’ as the key
So its actually Detroit SYSTEM that is the key to their success.Not the skill of individual players but the players ability to play the system.And, the System;‘s clear goal is to increase Corsi!. My guess is that they are doing yet ’unknown’ things to drive Corsi. I believe they consider the angle of their shots so that they recover rebounds. They purposely miss the net At home often to recover the shot etc etc.Now this explains there long term , consistent success. So Hawerchuck is correct to point out that Avalanche last year were the result of random variation and were a less than average NHL team.And, completely expected as an outlier given the luck/skill ratio in the nHL.
The key reason why Detroit focuses on Puck possession and Corsi is that
its very hard to influence anything else in NHL. For example, we know that drawing penalties leads to increase chance of scoring. Yet there is little correlation between skilled teams and increase power plays >Why? randomness of penalties both in calls and types of penalties (new rules) and bias of referees to call penalties for teams trailing ( see Objective NHL ). Hitting although wanted by fans and certain teams actually has a negative
correlation to win%.So does fighting. So does blocking shots.what else would you focus on if you were running an nhl team. I would simply copy the ‘experts’ Detroit!
at the same time Detroit have not had nearly the expected success in the playoffs.(Puck prospectus has shown that their regular season dominance would normally lead to better playoff success. Here is the randomness factor.Again expected given the small sample size of the playoffs
by dan lortimer on Nov 25, 2010 9:04 PM EST up reply actions
Really, I understand randomness. I really do. I’ve done high-level math, stats and physics. And I’m pretty sure that I was clear that the intangibles are not a predictable set of data. I’m pretty sure I was also clear about there being a problem with the (very good) model in that the quality of the team is determined recursively on a weighted set with a small sample space.
Your argument itself is flawed since you begin with a given that you pull out of your hat. But let’s go on. The Red Wingss’ system does drive corsi and It has been shown experimentally that it works for them. However, for any system to work, it involves personnel to adhere to the system — if we’re filing something and I start filing alphabetically and you’re filing chronologically, it doesn’t matter how good the system we’re supposed to be following is — it fails. This model, and your argument, do not take that into account.
You’re also making the assumption that the Red Wings’ system is the best system and therefore things that don’t look like it are automatically inferior.
I’d argue with the rest of what you said but there’s really no point. There are large, large holes in your logic and it’s kinda amazing to me that you’d adopt such a patronizing tone, implying that if I were “more evolved” I’d see the light, despite the fact that I very, very clearly stated that I liked the model and feel that it can’t really be improved upon with regards to predictability. The odd thing, to me, is that you seem to think the playoffs have a small sample size, when the teams are actually playing each other more often in tthe playoffs than the regular season so a direct comparison betweeen the two teams is much simpler and more rigorous to make.
Neurochemistry is not random though it is generally not conscious. Adhering to a system is as much a psychological feat as a physiological one. People are not robots. As such, there is a hole in the model that, unfortunately, cannot be filled. And I’m not using that as a loophole to throw everything out. However, I think it’s useful to discern brain states from random happenstance so that it’s clear what conclusions can and cannot be made.
Colorado was an outlier. Fine. The 80s Oilers were also outliers by this measure. That doesn’t mean that the model is wrong or fatally flawed. That doesn’t mean that the 80s Oilers were a bad team. That does mean that saying anything outside of “Colorado was an outlier” or “Colorado would likely do even better if they fixed this issue” is a jump that the math does not support — by correlation or causation.
PS: seriously, don’t treat anyone on the internet as an ignorant idiot. They probably aren’t.
I mean really, resisting reality? Give me a fucking break.
And I’m pretty sure that I was clear that the intangibles are not a predictable set of data.
A question: If the intangibles, neurochemistry, a roll of the dice, they’re all causal at a certain level. But if neither the person in question nor his opponents, nor his teammates, can reliably affect those factors, what’s the problem with calling it luck? In some sense, I’d say that luck is all the stuff that’s outside of your control; if the other team comes out flat, you got lucky. It almost certainly wasn’t anything you did or could predict with any regularity.
I tend to think of it in terms of a weather analogy: Climate is predictable, but weather as unpredictable past a week or so. The climate would be the underlying numbers; Corsi, drawing penalties, all that jazz. The weather would be whether it’s rainy or sunny that particular day — the team can get a sunny day, their shots go in and the opponents stay out, or they can get a crappy day when the opposition comes out flying and shooting the lights out. Regardless of weather, you still have to go to work and try to create felicitous results for yourself.
If we can’t reliably control our neurological states, or intangibles, I’m not sure how you can build around that or count it as anything but lucky when it happens to coalesce just so for your favorite franchise, or unlucky when everything falls apart.
"Essentially, all models are wrong, but some are useful" George E.P. Box
by Knee high to a duck on Nov 25, 2010 10:16 PM EST up reply actions
The roll of an unweighted die is, by definition, random and not causal. And the thing is that some people do have better control over their neurochemistry (see the Gretzky example).
If luck is anything out of your control, lack of preparation shouldn’t count as luck, but in this model it does.
I agree by and large about the weather/climate thing. I’m not saying there is no luck in sports. obviously there is.
But you can reliably control your intangibles. See Detroit’s System. Saying that how well you can control them is luck is a bit questionable.
I disagree; the roll of an unweighted die is affected by how much spin you put on it, how flat it’s tossed, et cetera. It’s definitely a model-able physical process, but the outcome is effectively random without extremely precise inputs. If we don’t have the equipment or ability to make those precise inputs, we can consider it random, or luck, for our intents and purposes, no? I think it’s a distinction without difference when it comes down to our results, much the same way that intangibles are.
"Essentially, all models are wrong, but some are useful" George E.P. Box
by Knee high to a duck on Nov 25, 2010 10:36 PM EST up reply actions
Right, but the thing that rankles Ender and me (and I’m sure a lot of the Avs fans Gabe’s talking to) is that, well, preparedness and adherence to the system and stress levels are not luck, any more than the ingrained motor-neural patterns that define “skill.” It is not a modellable variable, but it is not random, either, except insofar as the fact that we can’t define how it affects the model.
Maybe that’s a distinction without a difference for this purpose, too, but semantically, it’s always bugged me, and why I’ve been slow to accept “luck” as the entire answer.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
Adherence to the system is only as effective as the underlying system, no?
Another thing I’ve been thinking about is exogenous luck versus personal luck. From my perspective, the stuff I can’t control is luck, whether or not the process is random. Exogenously random or lucky? No. But from the angle of whoever or whatever is being analyzed? I think it’s fair to call that random variation.
I’m beginning to study econometrics; one thing that always has to be included in the model is the error term. That error term is there for stuff that’s not necessarily random, but is so complex or unpredictable that it’s not feasible to include it in the model. For example, if an apple-eating phage pops out of South America, the price of apples is going to rise around the world and the process wasn’t random, but it’s going to show up in inflation and there was no reasonable way of predicting that from a farm in Iowa. That apple farmer was very unlucky, even though the phage emerged through some causal process or another.
"Essentially, all models are wrong, but some are useful" George E.P. Box
by Knee high to a duck on Nov 25, 2010 11:07 PM EST up reply actions
Adherence to the system is only as effective as the underlying system, no?
If by underlying you mean overarching, yes. Definitely. And that’s only as effective as the confrontation between two teams’ systems with the level of adherence either team has on any given night. It’s not random or luck, but as far as the model is concerned I’m fine calling it noise or the error term. Still don’t like attaching the wordluck to it though.
Here’s my definition of luck. If I’m in net and someone dekes around me and hits the post, I’m lucky. That player isn’t unlucky — they just messed up. Using the word luck when you’re outside of the box (like we are discussing this) implies we know what was in the players’ control or not, which we don’t. It implies some sort of objective luck metric which we don’t have. Noise? Sure. Luck? Tough to say.
But that’s semantics. Like I said before, I just think it’s useful to keep in mind when making conclusions that there is definitely some non-random “luck” in there, and that should give your conclusions an error margin.
Sorry, I thought you were referring to the mathematical construct of a die. And a physicist would never go for what you,re calling luck here. If you don’t have the means to test something you know isn’t quite random it becomes a margin of error, not luck.
When it comes to the results of this model, you’re right – the nomenclature doesn’t matter. But separating non-random noise from random chance seems like an important distinction, whether or not it affects the math.
The odd thing, to me, is that you seem to think the playoffs have a small sample size, when the teams are actually playing each other more often in tthe playoffs than the regular season so a direct comparison betweeen the two teams is much simpler and more rigorous to make.
The problem I have with this is the transient factors that are present at any given time in hockey. Sometimes the coins come up heads, sometimes that fucker just keeps landing on tails (I’m looking at you, Jaroslav). Seven games (max) is an awfully small set when the number of expected successes is so small as goals. Toss in injuries, different referees, and a host of other factors that aren’t generally persistent and I don’t see a seven game comparison as superior to an 82 game comparison, even if they’re not playing exactly the same teams.
"Essentially, all models are wrong, but some are useful" George E.P. Box
by Knee high to a duck on Nov 25, 2010 10:23 PM EST up reply actions
I said a direct comparison between the two teams. No two teams have the same systems or the same strengths or weaknesses. For example, I used to play chess. Fairly bad at it. I’d win about 50% of the time. A friend of mine was really good. He’d win about 95% of the time, playing the same people.
But when I played him, I beat him 90% of the time. You can’t compare averages to specifics and expect real answers. The MTL/WSH series last year makes more sense when you look at their season series.
Dude, I’ve been over those seven games about a million times (it was a long damn summer) and the season series didn’t bear all that much resemblance to what happened in the playoffs. Washington was 3-0-1 against Pittsburgh the year the Pens won the Cup.
I said a direct comparison between the two teams. No two teams have the same systems or the same strengths or weaknesses.
Let’s say for a moment that this is what we’re going for: if you’ve figured out how to model the way that two teams interact, then you best not disclose it here and you best be on the next plane to Vegas, toot friggin’ sweet.
But even if it is, a comparison of two teams directly doesn’t tell you as much about the results each of those teams gets when facing other opponents as the body of work in the regular season does. The predictive value is much lower for a seven game sample than it is for an eighty-two gamer.
Chess is something I use as a baseline; there’s a game that’s as causal as causal gets, just two players on opposite sides of the board, no luck there. Except there seems to be. If it were pure skill, then you’d see the better player win 100% of the time — in the case of your friend, he was unlucky that you opened with the Reti system and he was ill-equipped to handle that. Or whatever. Even among IGM level players, you see significant swings in winning percentages that really shouldn’t be. Those guys are playing on such a high level that the one tempo at the start of the game essentially forces black to play for the draw and they’re not always able to get it. But IGM matches routinely end at 3.5-1.5 or worse. It’s not just matchups, it’s something they’re not able to persistently control. I call it luck.
"Essentially, all models are wrong, but some are useful" George E.P. Box
by Knee high to a duck on Nov 25, 2010 10:59 PM EST up reply actions
Again, semantics. And I’m not saying that you can necessarily predict the wins, but a series is a much better indicator of which is the better team of the two. The worse might be the best team in the league, but the worse of the two. It’s all in where you draw your sample lines. This just seems to be the only apples to apples comparison available. I’m fine with calling things undefined, but I call luck things out of the players’ control, and i personally don’t see that as the case here.
That doesn’t even pass the smell test. Across a smaller timeframe, “shit happens” allows teams to upset more often, for “luck” (or randomness, or whatever you want to call it) to win out. This has been show in best of 5 vs best of 7 series for example – you see more upsets in best of 5 series. Also, keep in mind, a best of 7 series can be over pretty quickly. Maybe one game a team gets a lucky call/no-call from the ref that is key to the game. Maybe the next game a puck bounces off a stanchion and surprises the goalie as he’s leaving the net to play it. Then maybe you just plain win a game on skill. Then maybe the next game the other team hits a few posts. Bam, that’s a series that swung on luck.
Remember, no one can guarantee themselves anything. “Shit happens” is an outrageously important factor in sports. The only thing a GM can do is build a team that gives him the best chance of overcoming “shit happens”. The best team still loses games, they still have random unlucky bounces. Just because “shit happens” doesn’t mean the team is actually worse than another team – and that’s what you’re saying, because in a best of 7 series, the sample size is small enough that “shit happens” can carry the day, despite a stacked deck.
One of the biggest upsets I’ve ever seen was the 2003 Ducks beating the 2003 Wings, en route to an eventual Cup final. I don’t think anyone in their right mind can sit there and say the 2003 Ducks were a better team overall than the 2003 Wings. Those Ducks just happened to get one of the single greatest goaltending postseasons of all time. Giguere was huge, at the right time. That was unpredictable, and that’s pretty much the definition of “shit happens”. If those two teams play a 100 game series, the Wings almost certainly win out – Giguere’s performance was sustainable over that period, and that was far and away the difference in the series. But across 4 games… “shit happens”, and it doesn’t mean Anaheim was the quantitatively better team – it means that Detroit was just not good enough to overcome that amount of luck going against them. Remember, the reason we watch sports is because the better team doesn’t always win. Sometimes, the best team even has a losing streak, but it doesn’t mean they’re not the best.
http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.
It’s the only apples to apples comparison available though. This isn’t baseball. Each team clashes systems and personnel 1,4, or 6 times a season, and rarely are the two teams equal on the injury front or even on the systems adherence front.
In the playoffs you have 4-7 games of the same two teams with the same rosters butting heads. Yes, the sample is small, so luck will be somewhat magnified, and no, this analysis has no predictive ability, but it is the most accurate representation of which is the better team of the two, against each other.
I’m not sure how apples to apples comparisons don’t pass the smell test, but meh. I’m also not sure why people think that posts are luck. Really, they’re not.
Why not look at what 7 games can tell us about individual players? Can 7 games tell us an individual player’s true talent, or definitively tell us that player is better than the players on the opposing team? If so, Ruslan Fedotenko should be a damn magician by now…
Does he call it Luongo underwear?
Writer at Behind the Net: www.behindthenethockey.com
by Bettman's Nightmare on Nov 26, 2010 10:09 AM EST up reply actions
Tango has a formula to calculate this -i’ll try to dig it up.
Basically, it calculates the following;
If you observe X games this the Confidence % you have knowing that the results are from ‘Skill’.
I believe for 7 NHL games its approx 12% .Therefore 88% chance results are from luck. This makes you wonder how many times GM’s’fansmake incorrect decisions on players over short periods of play. (i.e.; preseason, or one playoff run)
Colorado was an outlier. Fine. The 80s Oilers were also outliers by this measure. That doesn’t mean that the model is wrong or fatally flawed. That doesn’t mean that the 80s Oilers were a bad team. That does mean that saying anything outside of "Colorado was an outlier" or "Colorado would likely do even better if they fixed this issue" is a jump that the math does not support — by correlation or causation.
Who has said anything aside from “Colorado was an outlier”? Gabe’s whole point has been that they were an outlier – the team rode some good luck into the playoffs, when in all likelihood they should’ve finished out of the playoffs. Teams with that kind of Corsi rating generally are not very good and generally are not in the playoffs. The Avs made it, so they’re an outlier. Outliers are outliers because they’re generally a convergence of several lucky (or unlucky factors) coming together, which are not generally sustainable.
http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.
This:
the team rode some good luck into the playoffs, when in all likelihood they should’ve finished out of the playoffs
It’s one thing to say that a team which follows this path rarely makes the playoffs. What you’re sayin implies heavily that they didn’t deserve to make the playoffs because of this measure. I’ll bet this is what the Avs fans don’t like. And it’s not a statement that’s supported by the math.
The 1980s Oilers were not an outlier by this measure. They absolutely dominated their opponents – score effects are the issue.
Great point. I forgot about score effects. But my memory on that isn’t excellent. Pretty sure Bruce put it up though… Were the Oilers actually outshooting when they were down?
Regardless, again, i like the model, and I’ll be the first to say that it’s likely as good as it’s going to get. But I’ve still pretty confident that a team can drive what you’re calling luck over small sample spaces and if that’s the case, I think the word is misleading.
But I have two questions for you:
1)Is Colorado’s success through the first quarter of the season still a surprise/outlier?
2)Predictability aside, how do you feel about judging how good a team is by Bulding an average from a weighted set of games/conditions/divisions/scheduling over an n of 82? Or is it purely about predictability for you and that question is irrelevant?
Actually, hits have no correlation with winning, either way.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
Intangibles and clutch play?
“but there’s an amazing amount of real, numbered data that show that intangibles not only exist, but also that they have a direct effect which is most visible at high levels such as professional sports.”
Okay. why not show us this amazing data and HOW it directly effects NHL performance? And/or Show us that ‘clutch play’ effects NHL performance as you claim?
To have an effect means it influences winning and performance so if we can identify it , then we could use it to predict performance?
As Gabe points out 72% of all NHL hockey games are tied or within a goal and probably
more in the playoffs. So, when you consider that the team that scores first or wins the first game in a series has a high % of wining – almost every second of NHL is ‘Clutch Time’. So the notion if it exists is irrelevant.
Gabe and others have clearly showed Fenwick/Corsi Even Strength drives W%
B. Burke’s model for NFL when applied to NHL clearly shows Luck/randomness is a large part of NHL as well.
Not patronizing I was criticizing your attitude towards ‘luck’, Not you?! No need to vent your repressed anger.Why take it personal?
“You’re also making the assumption that the Red Wings’ system is the best system and therefore things that don’t look like it are automatically inferior.”
No assumption. Red Wings Win % is the best. This is empirical evidence. Seeing some of us are trying to understand ‘what makes teams successful’, and what drives w% its an obvious team to look at because of the large sample size we can limit the effects of randomness. Could it be that all the Red Wings are ‘deep breathing at the same time’ or
their neurotransmitters are the key, or the water in Detroit has ‘magic properties’ , perhaps?
But the most likely answer is their Corsi generating System.
by dan lortimer on Nov 25, 2010 10:56 PM EST up reply actions
1.) This Johnson dude comes off as a university student to me. Hard to pin down what level though. His ignorance regarding what correlation actually means screams undergrad but his penchant for taking a pretty common concept (Fenwick SH%, “finishing ability”) and trying to rename and recast it is very much graduate-like.
2.) Blah blah blah intangibles, blah blah blah prove it, blah blah blah full of bullshit.
3.) Shuffling cards is a deterministic process. Don’t know how those poker players live with themselves when they use the word “luck”. Hockey fans must just be more sophisticated.
Not a student, haven’t been for many years.
----------------------------------------------------------------------------------
HockeyAnalysis.com - Taking a Deeper Look at the World of Hockey
by HockeyAnalysis on Nov 25, 2010 8:36 PM EST up reply actions
Funny thing: I learned just today in my motor control class about the connections between the entire frontal lobe — including the limbic system, which defines emotion — and the thalamus, and how these can interact with motor signals to affect output. We didn’t get into the gory details, but the take-home message was, your state of mind is directly tied to your ability to execute a motor command. (See also Ender’s original wall of text. Science: it works, bitches.) It may not be a big factor, but if this blog and others like it have established anything over the years, it’s that it doesn’t take big factors to swing the results, especially in the really small samples (i.e. playoff series and smaller).
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
Addendum: if you want something more specific on mirror neurons and your unconscious mind (the “zombie” Ender mentioned), you might try looking up V.S. Ramachadran, who’s done a lot of really neat work on the quirks of perception and consciousness. There’s also an episode of Nova with him on it, called “Secrets of the Mind,” worth checking out.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
Interesting discussion, and I don’t know where to reply, so I’ll just randomly chime in here.
When I talk about the luck component in the game, I’m using Tango’s formulation:
std (random) = sqrt(0.5*0.5/82) → that’s the stdev of a binomial random variable over 82 games, which basically models a league full of identical teams
And then std(talent) = sqrt(std(observed)^2 – std(random)^2)
It’s essential that we include this component – it incorporates the difference between team true talent and observed performance.
The rest is true talent, and that incorporates a lot of things. It manifests itself as shot differential, goaltender talent, special teams talent and shooting talent. But at a more base level, what drives this talent? Certainly fitness, mental readiness, focus, coaching, systems, and of course hockey talent. Does it really matter how apparent talent breaks down between those factors? Yes, but when we’re evaluating a team’s true talent, all of these things are lumped together, and they result in good corsi or special teams results or good goaltending and we incorporate the tangibles and intangibles in the model.
Bottom line: you can’t ignore that big chunk of luck. It really is sampling uncertainty, not unanalyzed intangibles.
I could be misreading you, so I apologize in advance if that’s the case, but your second equation seems to only check to see how much better or worse the team is being observed rather than an expected norm. Not sure how that equates to luck. Seems to indicate “something important is happening here” for sure, but I’m missing the luck bit.
Am I just misreading you?
It’s a formulation for separating signal from noise. We see a process with variance VO. We know that there is some amount of luck involved, since we can’t predict convincingly who will win any one game, and we know how much that luck would be over 82 games if all teams were equal. So the final variance is the sum of the “talent” variance and the “luck” variance, since they’re not (in the long run) correlated. So if we assume observed variance VO, luck variance VL and talent variance VT, we have VO = VT + VL.
Thus VT = VO – VL.
Variance is just stdev^2, so stdev(talent) = sqrt(stdev(observed)^2 – stdev(luck)^2).
In practice, the luck component is slightly less than a binomial, since there is always a team that is slightly better than the other, but it’s so close that the estimate is good enough.
Does that make more sense?
Wait, actually, scratch that. Are you saying that the “luck” variance is just the variance you’d have with a normal curve all else being equal? If so, I hold to everything else I’ve written here, because all of the things I’ve mentioned tie directly into that.
It’s the variance of a binomial, but otherwise yes. But I read everything you wrote, and there’s no conflict: unless you think that a player can get “in the zone” enough to make him team’s chances of winning 100%, and I doubt you think that, then there’s still a random element involved. Sometimes shots just don’t go in, sometimes they do. Just because the binomial is 60/40 instead of 50/50 doesn’t change the importance of luck much.
I agree for the most part. That said, i still do think that style of play can drive luck — if you get the puck deep and keep it around the net it doesn’t matter if you get shots on goal. You still increase your chances of a lucky bounce leading to a goal. It’s still luck, but i don’t think it can be divorced from play.
But that doesn’t affect the math — i just think that disregarding that is likely where the furor comes from.
If clutch existed and muscle control really was elevated by state of mind and other such flaky factors you’d think the first place you’d see it was in clutch goals.
Well, turns out, Pythagorean W% explains a whole bunch of W% and clutch matters a whole lot less than scoring more goals than you allow.
Weird.
Clutch might matter more in Rock-Paper-Scissors though.
by R O on Nov 26, 2010 1:35 AM EST via mobile reply actions
You’ll have to elaborate on that. I’ve never seen any evidence that there are players who are “clutch” beyond what we might expect from random chance alone.
I don’t doubt that there are players with better abilities in pressure situations, it’s just that no one can identify them prior to them doing something “clutch”. If you can’t predict something then it doesn’t have much value to someone running a team.
What I think Ender’s saying (and he’s welcome to correct me) is that just because you can’t model “clutch” doesn’t mean it doesn’t exist, as R O postulates. It’s quite obvious that it exists, and there are scientifically valid neurological reasons for it to have an influence, but that doesn’t mean that it’s something you can readily account for or predict; who are we to know an athlete’s mental state before or during a game? In terms of modelling the game, the distinction may be practically irrelevant, given the data available, but it’s always worth bearing in mind that not everything we can’t predict is “luck,” in the general understanding of the word. (Which may be why you’ve wound up in so many fights with Avs fans since the start of last season.)
Or, put more succinctly, R O might want to stop calling bullshit on something he doesn’t understand. Especially considering the very nature of “clutch” is that it’s a small-sample effect (it’s primarily invoked in playoff scenarios), and therefore Pythagorean W% is kind of irrelevant to the point.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
The Avs fans are mad because, as they said, they “defend their boys.” They already know the difference between talent and performance and readily own up to it (I can’t count how many hostile Avs comments begin “Everybody knows the Avs got lucky last year” and then proceed to blast me for claiming they were lucky) but don’t like it when I say it or Derek says it or anyone else not connected to the Avs says it.
So here’s the thing: if nobody knows who’s clutch (Tom seems to have shut Sakic and Roy down) and we can’t measure it and the experiments we construct show that it has an inconsequential impact on the game, why do people keep coming back to it as though it explains 100% of outcomes? How can people hand-wave away something like Corsi that has huge predictive power while clutching at Shot Quality, which demonstrably has only the smallest role in the game?
As for luck, do you dispute the coin-tossing experiments that Vic and Likens and Tom Tango do?
I think Ender’s comment below mostly covers it. Again, it’s entirely a semantic distinction, but it’s that semantic distinction that’s probably getting you yelled at. Non-random + unpredictable != random, yet by calling everything in the error term luck, by the conventional usage of the word, that’s what’s being implied. Thus why people try to use clutch to explain more than it possibly can: it’s a more comfortable narrative than the implicit alternative of “random chance,” especially when it has a historically-recorded impact, such as changing the outcome of a playoff race or series. I mean, I suppose you could call the Eberle-Hall goal last night “clutch,” but it was completely fucking irrelevant, because that team is still garbage, so no one cares about one marginal point in the standings when you’re probably going to finish with 60.
Hell, in last year’s playoffs, I wanted to believe that the Habs were doing something special to win those two playoff series, but the bottom line, which I always understood in my rational mind, is that they were getting their asses handed to them by possession and chances and Halak and Cammalleri were carrying the team on the basis of their world-beating streaks.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
I know, right? You can’t teach that kind of clutchiness. ;)
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
To clarify the above comment about the Eberle-Hall goal…regular-season games in late November are not the sort of pressure situations that cause people to mythologize events. You could say that it was clutch, but I’d tell you that it wasn’t really that important of a game, given the state of the team and the time of the season, so it was just plain old good timing.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
Clutch is unpredictable, dubious whether players even control their own clutch ability, yet it’s vitally important to winning hockey games.
Give me a break. As far as turds go, that’s sizable.
Clutch goals, timely goals, they should allow teams to outperform their goal diff, and persistently so. Otherwise what’s the point?
But since the lockout every team season has looked like it came from a parallel universe where goals come randomly and you score when you score and you don’t when you don’t.
Either that’s a monstrous coincidence or we live in that universe.
You and Ender are oviously bright but you’d do well to apply yourselves to the real world.
by R O on Nov 26, 2010 4:33 PM EST via mobile up reply actions
When did either of us say that it was vitally important to winning hockey games?
I mean, nice straw man there.
So again, open a book if you want to see whether the idea of clutch is possible or probable. You might even learn that it’s inherently not predictable. But it most certainly is not random chance.
If applying myself to “the real world” requires me to throw out what I know about mitigating factors and saying “i don’t have a way to handle this so I’m just going to say it’s random because it’s indescipherable to me” I’d rather not. That sort of thinking limits the understanding of the model, and, while mathematically equivalent, is factually incorrect.
Seriously though, when did the real world become a mathematical approximation rather than what actually happens in real life?
when did the real world become a mathematical approximation rather than what actually happens in real life?
Not sure where you’re going with this. All of this analysis is trying to project a team’s performance in the 2nd half of the season or in the playoffs. A predictive model is necessarily a mathematical approximation, and after the fact, you look at the real-life events you were trying to project and see how well you did at projecting them.
The luck we keep talking about is the sampling uncertainty in the observed results. Even if we knew exactly how good every team was, we still couldn’t predict their records exactly – there’s luck in the outcome of a small number of trials.
Even though I accept that “clutch” must exist in some way, I can’t quantify it in a way that improves my predictive modeling (past attempts to quantify it have not shown it to be a significant true talent.) I’m certainly always looking for experiments to find additional drivers of team outcomes, but nothing makes it into the model until I’ve found evidence for it.
And I completely with nearly everything that you just said. The title of your comment is directed entirely at R O, nobody else.
I agree that there is some luck. I agree that there are some non-random factors that are not readily quantifiable/predictable, and you can’t pull them into your model in any meaningful way.
All I’m saying is that these things definitely exist, whether they’re visible in the math or not. Few people would argue with that. But when you lump it together with true random chance and call it luck, you jump from saying “the avs got into the playoffs despite our model predicting otherwise” to “the avs shouldn’t/didn’t deserve to make the playoffs.”
Yes, it’s semantics, and I know since you’re familiar with the math you know where the limits are whether you expressly state them or not. But I’d expect that you get a lot of Avs fans who think you’re making the judgment statement of deserve due to your use of the word luck to handle both random and nonrandom things.
And I think that’s where the verbal/scientific disconnect lies and why you will have avs fans who are willing to say they team got lucky but aren’t willing to hear someone say they should or shouldn’t have made the playoffs.
Because the deserve or should statements is not supported by your math. They can’t be. It’s the nature of predictive math. “We wouldn’t expect” is not the same as “This shouldn’t be happening.”
And just so we’re clear, I’m not saying that you do this. To be honest, I haven’t read enough of your stuff, but from what I have read, you’re pretty unbiased that way. But a lot of people are making affirmative statements regarding things (Derek Zona, for example) that they don’t have the grounds to make and that’s reflecting on you as the math model guy.
I have a feeling that most of the furor coming your way is due to how other people are analyzing and stating conclusions based on this data.
If you look in the comments on one of my previous pieces, you can see that someone else says they have a model that shows the Avs were a .520 team and the echo chamber doesn’t question him, and one guy says he looks forward to reading his future work. So the problem is not with the model’s predictive value.
Honestly, this standard is only applied to corsi analysis. I’ve published a bunch of horseshit pieces about shot quality and goaltending that I’ve come to realize were completely wrong but no casual fan ever said “Hey Gabe, your model has no predictive value and you can’t make conclusions based on it.” (Vic said it was horseshit, but only after running the numbers on it.)
I never said there was any issue with the model’s predictive value. I think I even said that it’s very close to the best we’re likely to ever get.
Either way, I personally wouldn’t call your model shit without running numbers.
Just trying to explain where the speedbump is and why people take issue with corsi. I think it’s mainly PR and I think the PR issue is twofold:
1) People make affirmative/judgment statements using Corsi/Fenwick to say something that the math does not support
2) The word “luck” is a bad word to use. Say “luck and other noise” or something, but when you use the word luck it implies random or — at the very least — something out of everyone’s control. And some of what you’re calling luck is exactly that. And some of it is not. Do we have a way to say what proportion of either is? No, and we likely never will. And that sucks because it would shut a lot of people up.
That’s all. I think you do great work and while I disagree with some of the premises and methods of your math, I admit it’s a great model and taking the other things into consideration would more than likely reduce the efficacy of the model because it would make the n of 82 way too small to work with.
But if you want to know why people get angry with it, is there’s a hole that few of the math people seem to want to acknowledge (luck) and since that’s not something that can be dealt with via the math, you have a lot of people who feel something is finite and important while the stazis tell them it’s unimportant and it’s just plain luck.
It’s semantics and PR, but if you don’t want to qualify your terms a bit tighter, you’re just going to keep getting what you think is irrational rage.
Just giving you a why. Not saying you should change your work or that your math is wrong — just pointing out a limitation in it and telling you the semantics regarding that limitation is likely the source of your public’s issues.
Vital to winning hockey games on a regular basis? Of course not. No one’s that stupid.
Occasionally a difference-maker, in a single game? Sure. Not exactly going to budge the needle in the standings, which are based on 82 games: I’m not convinced there’s any such thing as “clutch” in November, anyway. But when that single game happens to be the deciding game of a playoff race or a playoff series, the ability to not fuck up under pressure is a relevant skill. (Done any public speaking? Ever tripped all over your words? Yeah, it’s like that.)
Because of the small samples involved and the relative historical impact, it’s relatively easy to throw out 82 games worth of data if you get four good ones in a row. That doesn’t mean that those four games were bullshit or luck. If a player or group of players is on their game, they can play above their “established” level of talent for a brief period. Happens all the time. Don’t see why that always has to be bounces, or exclusively be bounces. Not when there’s perfectly valid science showing a link between state of mind and motor control. Doesn’t explain all the error, and it’s not possible at this point to say how much of the error it explains (and may never be), but in terms of how the real world works, you can’t say it doesn’t exist. You can say it’s not mathematically relevant, but you can’t say it doesn’t exist.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
I’m not convinced that we can take our experiences as amateurs (public speaking, beer league, high school football, little league baseball) and extend that to people who’ve done 100k+ reps at their day jobs.
The spread of psychological readiness among pro hockey players is as small as the spread of their body fat percentages. And the biggest mental error of last season, from my perspective, was Martin Brodeur in the first olympic game against the US. Nobody would ever suggest Brodeur is anything other than a consummate athlete, but he made a ridiculous mental error.
Also, out of curiosity, you say that you don’t think we can extend our experiences to them, but follow with
The spread of psychological readiness among pro hockey players is as small as the spread of their body fat percentages.
So we can extend only our best attributes to professional athletes then, and professional athletes are all effectively robots?
Just kinda wondering if you have some sort of citation or reason for that quote.
I’m a kinesiology major, and I think the most compelling evidence I can offer in that respect is a study my exercise physiology prof, of all people, cited, effectively showing the placebo effect in elite cross-country skiiers. These are all people with VO2max values in the 70s and 80s (average is closer to 40), who have trained for dozens of years to do what they do, and run hundreds or thousands of races, and yet there we had this placebo (compressed air puffer) giving the same results as an actual tested drug (a vasodilator, essentially an asthma inhaler). Both improved race times by the same amount.
Speaking more generally, I really don’t see any reason to dispute that elite hockey players, while they’re better than us at everything in the sport, are still human and still subject to human flaws. Less susceptible? Of course, for the reason you’ve stated. Not even Vesa Toskala’s gonna shit bricks and give up six goals on eight shots after letting in one brutal shot like I’ve done in the soccer net. But I see no reason to think that a player or team wouldn’t get tight and start trying to do too much when they’re down by two in a playoff game. Again, we see that all the time.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
IMO, clutch matters and exists. A lot. There’s just no way to know who’s feeling clutch on any given day, though I wouldn’t bet against Patrick Roy or Joe Sakic.
by red army line on Nov 26, 2010 7:54 AM EST up reply actions
SNNEnder, I agree with you that neuroscience applies, and having already been in “the zone” myself (not in hockey, but mentally) I will attest to its existence. But the fundamental question is if it’s repeatable or predictive? If it’s not, then it’s luck by any other name. If you could show me that some players can get demonstrably clutch or unclutch at specific times, beyond what we would expect from data mining, I’d buy it, but so far everything points to the fact that clutchiness is used as a catch-all explanation after the fact: post hoc, ergo propter hoc.
Again, if you have data to the contrary, I’m all ears.
Has anyone looked into Patrick Roy or Joe Sakic? To my eyes they seemed to have an uncanny ability to (for Roy) pitch shutouts or (for Sakic) have multi-point performances in important games.
by red army line on Nov 26, 2010 10:02 AM EST up reply actions
You’d have to identify “important games”first, and then make sure that it was significantly different than their “normal” game performance. Honestly, both players were so good in their own right it’d be a tough one to make convincing.
Does he call it Luongo underwear?
Writer at Behind the Net: www.behindthenethockey.com
by Bettman's Nightmare on Nov 26, 2010 10:17 AM EST up reply actions
I think on NHL.com or on NHL Network they ran down a list of “top clutch players” and finished with Roy #2, citing his 40 playoff OT wins (I think…it was 40 playoff something…and 11 playoff OT wins in a row or whatever it was in his rookie year is pretty amazing, and I guess those three Smythes should count for something, since not all Roy’s teams were loaded), and Sakic #1 with 7 playoff OT goals (plus I guess a 3 point gold medal game in 2002, 3 point Game 7 in 2001; I don’t know his history much more than that, though).
That’s a good point though, I guess their “normal” performances were so good that their “clutch” performances weren’t all that far from normal, at least not far enough to be deemed so improbable that they need to be labeled as “clutch.”
by red army line on Nov 26, 2010 10:36 AM EST up reply actions
You know you can’t mention Roy and “clutch” without someone pointing out the Statue of Liberty, right? So I’m mentioning it. Just because it makes me feel good.
A couple pieces from TCG at BrodeurIsAFraud on the subject of playoff goaltending and “clutch-ness”:
Clutch Play
Which Goalie was best in the clutch
http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.
Thanks for the links. I got into reading these blogs a bit late and haven’t gone through all the old posts yet.
by red army line on Nov 26, 2010 5:48 PM EST up reply actions
It was 11 OT wins in a row in the early 90s (specifically, 10 in 1993 and one more whenever he next had a playoff OT game).
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
I hontesly think with Sakic and Roy, you had players who are great players all the time. Roy had his fair share of stinkers in big games, but he was a great goalie, pretty much all the time. Sakic was one of the best players of all time.
They weren’t clutch, they were just damn good.
Your mama so ugly they call her "The Avs Powerplay"
Jibblescribbits: C'mon over and waste some time
by Jibblescribbits on Nov 26, 2010 11:14 AM EST up reply actions
While I know this is far from conclusive, Sakic’s stats are:
Regular season: 1378 GP, 1641 Pts, 1.19 PPG
Playoffs: 172 GP, 188 Pts, 1.09 PPG
Taking into account the tougher level of competition in the playoffs, it sounds like a wash to me.
I’m not debating that Patrick Roy isn’t the most dominant playoff performer in recent history, but it’s not like he was a crap regular season goalie either. Pitching shutouts? In the Canadiens’ two Stanley Cup seasons (1986, 1993) he got a grand total of 1 shutout. He got 7 in the Avalanche’s 2 Cup seasons, but that was the deadpuck era; for comparaison, Hasek got 6 in the Wings’ 2002 Cup year alone.
Another useless stat? The Avalanche won 2 President’s trophies (1997, 2001) and 2 Stanley Cups (1996, 2001).
I said earlier that it’s not predictive.
I also said that non-random non-predictable effects are not random chance, so lumping themselves together and calling them luck might work for the mathematical model might work, but it becomes an error in the theoretical model.
That make sense?
Players like Sakic and Roy likely lighty weigh the “luck” component in a non-predictable way in games we feel are important. This isn’t going to show up in the math because it is likely a very, very small effect which isn’t predictable.
You won’t see me label a player as clutch, but I think it’s silly to say nobody thrives on big pressure.
I think we’re converging here. I, and most reasonable people, will agree that the effect exists. Observation and mathematics tell us that the effect is small.
The bottom line is: if the effect is small, it’s not likely to be the reason for our model and reality diverging. Random variance is much larger than psychological effects. That’s all we’re saying. People like to cite clutch as the reason why something improbable happened; the odds that it was clutch, rather than randomness, are infinitesimal, given how little “clutchiness” shows up persistently in the results.
And I just don’t think you have the grounds to call it infitesimal. It is likely small, but as I said above I don’t think you can divorce luck from play because while you can’t determine luck you can still increase the chance that luck will lead to an actual goal rather than just complete a pass that otherwise shouldn’t have been completed. And on the small sample of one game, that definitely matters.
Bt I do agree that the model is very good and it has shown it’s own predictive power. Just pointing out a couple of things that haven’t (and as far as I’ve seen can’t) been bourne out by the math. And I think thats largely the reason why people get angry with it. When it’s used to make affirmative statements like a team shouldn’t or didn’t deserve to win (not that you’re saying that, but some elsewhere definitely are) and these things can obviously be mitigating factors, it builds rage.
But you’re right. For the most part the two of us do see eye-to-eye, and I’d argue that for the most part everyone does. But i don’t think a certain level of animosity at how corsi has been presented is irrational — there are limits and mitigating circumstances that aren’t often stated, and the language is often used in a way that it shouldn’t be (on both sides).
I thik the big reason that people lack onto clutch is that they’re backed up against a wall (because the math is good) and need to grab onto something because they feel that there is something else going on. And there usually is. This will show itself more readily in small samples and I get that with a large enough n it should even out. But that doesn’t mean that a team can’t be the masters of their own destiny by tipping the odds of luck in their favour.
Really, people should just be more careful with how they word their conclusions. I’m betting that you know the difference between “we wouldn’t expect” and “this shouldn’t be happening” but there is an increasing number of people on the internet using tand talking about this data who don’t.
You seem to agree clutch cannot be predicted
So why is it outrageous to say “clutch performances” are random, aka “lucky”, therefore fall into the “luck” column. What luck mostly means is things like illness/being in the zone, emotional weakness/strength, fatigue/rust/rest/hot streak, etc, that are all mostly explicable, but not really predictable.
SCHIENDER FOR VEZNA!
FWIW
I’m pretty much on board with Corsi analysis, but I’m still calling you guys The Corsiatti because
a) I coined it
and
b) I still think it’s a funny name.
Your mama so ugly they call her "The Avs Powerplay"
Jibblescribbits: C'mon over and waste some time
by Jibblescribbits on Nov 26, 2010 11:16 AM EST reply actions
I’ve been calling the Oilblog stat guys the Edmonton Eulers for years.
SNN Sports - A theoretical Oilers blog (i.e. theoretically, I write stuff there). Link now 100% less broken.
My one complaint with Corsi
My one complaint with Coris is the name. I know it’s named after Jim Corsi, but I can’t think of a dumber reason to name a statistic. (I do long for the days of the Adams, Norris, Patrick, and Smythe divisions though, so I may be a bit of a hypocrite here)
Almost every other statistic’s name bears some information about the actual statistic. Corsi does not. I find it almost humorous that a statistic that explains a whole lot about hockey has a name that explains nothing about the statistic.
Would it be that hard to use the terms “attempted shots +/-” for Corsi and “effective shots +/-” for Fenwick.
Personally, I do think the name actually fuels some of the debates that come up from the anti-statistics crowd. For those that don’t like stats, it reads a little bit easier to say,
“Our current stats, like goals and +/- are where it’s at … these new-fangled Corsi stats are just mumbo-jumbo”
than to say,
“Our current stats, like goals and +/- are where it’s at … these new-fangled attempted shots +/- stats are just mumbo-jumbo”
Yes, I realize it’s the same thing being said, but I still think the name does have an impact on people’s reaction. They may feel like something is being hidden with a name like Corsi.
Yes, framing and marketing are big issues.
But why do a bunch of guys who write about hockey for free (or nearly free) need to focus on such issues? The people you see writing about Corsi work for pro sports teams and have careers as professional gamblers. If these people are making a living inside the game on what they know, why assume that they’d want to expend effort becoming low-paid entertainers?
The press
already uses those terms you cite, or paraphrases. I’ve seen vancouver newspapers use corsi several times without calling it that. In fact, Ron McLean is the only MSM guy I’ve heard use the word.
As Gabe, Tom, Vic, and all the other corsiatti make an impact on MSM guys reading their work (Mirtle is gonna be gold in 5-10 years), the terms will likely become more accessible.
SCHIENDER FOR VEZNA!
The Effect of Schedule
I wonder – as the Avs played in the weakest division in the West last season and given the schedule still favours inter-division games, does this account for a significant amount of the luck that exists in the model in this instance (if it indeed able to be effectively incorporated into the predictive model)?
Differences in schedule difficulty between teams aren’t all that meaningful over the course of an entire season. Case in point, here’s how Colorado’s 0910 numbers change once a correction for schedule difficulty is made:
Corsi percentage: 0.446
Corsi percentage corrected for schedule difficulty: 0.447
Goal percentage: 0.512
Goal percentage corrected for schedule difficulty: 0.517
Seems as though the Avs actually had a more difficult schedule than the average team (most Western teams did).
but the important thing is not the “average” team – Colorado doesn’t have to fight the Eastern teams for a playoff berth. The important thing would be Colorado’s strength of schedule vs the rest of the West.
http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.
What about your site?
Hi;
Good stats. However, you have made a point on your site (which is great BYW) to
make an adjustment for Corsi (when the teams are close) for
Strength of Schedule.Am I missing something? Why do this if overall there is little effect?
Thanks Dan
As far as “clutch” players goes in the present day, Johan Franzen is a guy usually given that moniker by the media. And, in 3 straight playoff runs he has exceeded his regular season numbers. How many more games would he need to play before anything like that becomes statistically significant?
Part of the thing with Franzen is if you look at his numbers, he puts them up in bunches. In the DET/COL series a couple years ago, he had 10 points in 4 games and 3 GWG making Jose Theodore his bitch. He had 18 points total in the playoffs, 5 GWG. 18 points in 16 games looks good. 8 points in the other 12 games (all of which were considerably more competitive than that DET/COL series) is good, but nothing terribly special. That really doesn’t speak to his “clutchness” – the situation was anything but clutch, the game/series situation being what it was.
It’s all about context. Media is often trying to spin a particular story or explain something with a more interesting narrative than “shit happens” or “he got lucky”. That kind of context is often lost because its complicated or not interesting. To hear announces point to his 18 points in 16 games or 5 GWG without that context, well, its only telling part of the story. Not to take away from Franzen’s feat – 10 points in 4 games is pretty damn good, regardless. Franzen had a great postseason that year, but you have to look at the whole picture, and the whole picture shows that Franzen really stocked up a lot of his impressive numbers in the most lop-sided series of the tournament that year.
http://sacrificethebody.blogspot.com/
Sacrifice the Body - Examining the NHL through statistical analysis, reasoned thought, and blind conjecture.
by IAmJoe on Nov 27, 2010 4:17 AM EST up reply actions 1 recs
I look at the series with SJ last year and I see Detroit losing four one-goal games and Franzen scoring four goals in the first period of the blowout and only one other goal in the rest of the series. Isn’t that unclutch?
Well I assume everyone’s perception of “clutch” is high-leverage situations but in the example of Franzen you just gave if the game was close even early on, in the playoffs that is a situation of moderate leverage but as he keeps scoring those goals become less and less clutch.
I of course think none of this matters or is significant.
The Sports Psych guys are missing a key point here. The unpredictability or variability in several aspects of the game (SH%, deviation of W% from Goal% etc.) is already quite well modeled by weighted coin flips.
ACTUAL stats and probability guys can now jump in and say whether that’s binomial or hypergeometric or what have you, I’m no stats guy. But the concept of weighted coin flips (or if you prefer dice rolls) is, I think, easily understood.
So Psych guys are either arguing that the mental state is the physical process behind the coin flip, or they’re arguing that the mental state is responsible for the small differences between the coin flip model and the actual results.
The former is irrelevant.
The latter is irrelevant.
Wrong Again
The sports psych guys are actually stats guys, since you haven’t figured that out yet. They’re also not arguing that sports psych is something that is readily picked up by the math nor are they saying that sports psych is the reason for variation from the expected.
What I (and likely Doogie2k though I don’t want to put words in his mouth) are saying is that the fact that these things exist means that they can affect games on a micro scale, and so it is possible — however unlikely — that those micro effects can add up to put a team well outside of the model’s expected values on a macro scale.
These are just reasons why expected might not equal observed. No reason to throw out or even modify the model.
It follows, then, that if you are talking about these finite, real effects as a term that is lumped in with “luck,” especially when people tend to assume the word luck means random or out of your control, people are likely to feel that you’re misrepresenting the facts. When you shoot these people down by saying something like “it’s as good as luck anyway, according to the math, so why do we care?” you’re going to get a lot of angry people.
That’s all.
Hawechuk wrote this post as a rebuttal to people’s anger, which, as he sees/saw it, was irrational. All I’ve been trying to say is that the anger is rational, and these are the reasons people are getting angry.
You seem to think I’m arguing against Corsi or Hawerchuk or something, or even that sports psych is the most important factor in determining success. I’m not saying that. I never was.
I guess one could even say that your point and all of your comments on this topic have been irrelevant.
those micro effects can add up to put a team well outside of the model’s expected values on a macro scale.
You should talk about which teams have stepped “well outside” the simple weighted coin flip model of luck and in what way they have done so.
It follows, then, that if you are talking about these finite, real effects as a term that is lumped in with "luck," especially when people tend to assume the word luck means random or out of your control
According to you these are small-scale micro effects. Teams can’t repeat them, can’t predict when they will happen, can’t predict who will step up so to speak.
So that’s basically a reasonable facsimile of luck right? Out of the player’s control, since they can’t, um, control it?
You seem to think I’m arguing against Corsi or Hawerchuk or something
It’s hard to follow your train.
Psych effects are real? Ok so they ought to have some repeatability. Not much real estate for “repeatability” when variations from the repeatable parts we’ve already extracted (driving possession forward, generating scoring chances, PP, goaltending ability, the little bit of shooting ability that exist) look a lot like coin flips.
Psych effects are small? Right with you there. They occupy a small patch of real estate between coin flips and the real world. Small enough that it’s really not worth it yet (maybe not ever) to think about it, when there’s all the low-hanging fruit that some coaches and GMs still need to pick (i.e. matching a fucking line properly every now and then, not playing sinkholes like Bertuzzi, not signing the Nabokovs and Toskalas of the world).
I don’t know, I’m not a huge hard math guy and maybe that means I’m just not sophisticated because I rely on simple concepts like “coin flips” and try to make connections with what’s happening on the ice, instead of trying to characterize this as some abstract mathemagical exercise.
Maybe that’s why you get confused by my shit?
I’m not confused by your shit. Not at all. You’re confused by mine. If you want to know why what I’m saying makes sense, read a stats/econ/neurosci/neurology book. If not, well, I don’t really care.
I hear you RO. Ender seems to be like a dog chasing his tail.
However, giving his argument the ‘principle of charity’ we can say-
That the 65% non-demonstratable skill in the NHL is
unknown! (It is undeniably made up of a large amount of luck BUT ‘possibly’ other unknown skill factors)
If we assume the best case scenario that part of this is because of neuroscience
of certain players on certain teams.
We have to ask is it just some players on a team . Is it all the players?
Most importantly what part of the game is it improving -is it improving shooting%
and save ? But wouldn’t it also improve a player/teams ability to gain possession of the puck as well?
As an aside in Vancouver here there is this great debate about Henrik Sedin only having 2 goals this year yet he has just about the same amount of shots. No one can explain it. Henrik doesn’t know the team doesn’t know the broadcasters nor the fans. Hmmm.. seems as if Hawerchuck was all over this! Of course his S has simply regressed as he isnt getting the luck he had last year! I guess his neuroscience work isn’t helping this year?!
One last time:
1) Psychological/neurological effects do exist in real life.
2) Players are people, and so these things affect them a varying amount which is not predictable to an outside observer.
3) 1 and 2 might not be distinguishable from random chance in a mathematical model, but they are still not luck.
4) Some nonrandom factors can affect the effect of luck – keeping the puck down low increases the chances that a lucky bounce will result in a goal rather than just a completed pass.
5) Most people think that “luck” means random chance.
6) Treating nonrandom things as luck will result in many people scratching their heads.
7) Many people who are confused get angry at the source of confusion.
8) All (and I mean all) of this could be avoided if people took the time to make sure that the math supported their conclusions. There is a big difference between “we wouldn’t expect this to happen” and “this shouldn’t be happening” and the second you’re telling people that their team didn’t deserve to make the playoffs, you’re going to get anger.
Once again, since apparently this isn’t clear enough for you and R O: I WAS NEVER SAYING THAT THE MATH WAS WRONG OR THAT PEOPLE SHOULD LOOK TO PSYCHOLOGY TO FILL IN HOLES IN THE MATH.
There are just reasons why the math can’t explain everything and referring to anything that is unexplainable as “luck” is bound to raise some ire. When you do, you are telling people that something that is Real and True (with a hell of a lot more data behind its existence than a hockey model) doesn’t exist and wouldn’t be important if it did. I was doing nothing other than explaining why the anger likely exists and why it’s justified when discussing this topic with people who make affirmative statements like deserve and should.
I really don’t think it’s that hard to grasp. But hey, that’s me with my apparently fancy book lernin’
The real reason Avs fans hate being called lucky.
‘What else can you do?’ Over time they’ve proven that it’s not luck. They’re a very skilled team."
-Derek MacKenzie of the Columbus Blue Jackets talking about the Detroit Red Wings
via kk
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
