If you've ever visited Behindthenet.ca, you'll notice that I don't lump all even-strength or man-advantage play together. Instead, 5-on-5 play is separated from 4-on-4, and 5-on-4 and 4-on-5 are separated from 5-on-3 and 3-on-5, which I find wholly justifiable based on the vastly different shot rates at each strength, and in the case of 4-on-4, the higher leverage and random outcomes of overtime. This has drawn the ire of some out there:
"[N]ote that Desjardins has issues with the way he presents his numbers. He does not include 4on4 play at all. And, he does not present numbers compiled for all situations. This severely compromised his stats with regards to Ehrhoff who excelled at 4on4. If there are other shortcomings in his approach, please let me know.
No problem regarding the issue of sample size, but it would be incorporated if he did an all-scenarios numbers. Desjardins also omits 3on5 and 5on3. In all of these cases, there is a good case to be made for publishing the numbers despite the sample size. In Ehrhoff's case, he stands out as we ran the numbers separately. I can also see it in his play in that scenario last year and this year. Additionally, Ehrhoff got significant 4on4 TOI which mitigates the sample size issue. 5on3 shows a big deficiency on the Sharks, not singling out any player."
First, let's look at the percentage of total shots recorded at each strength last season, both league-wide and by Ehrhoff:
League-wide, 95.1% of shots were taken at 5-on-5, 5-on-4 or 4-on-5, with the other shots mostly distributed across two-man advantages and 4-on-4. A defenseman who plays 20 minutes per game might average 20 seconds per game at 5-on-3 or 3-on-5, and at 4-on-4 in regulation and in overtime. The NHL does not accurately record ice time at these strengths, so a 5- or 10-second error can completely distort a player's rate stats. In addition, 20 or 30 minutes per season at these strengths has basically zero predictive value. So, between uncertainty and high regression to the mean, I don't agree that "there is a good case to be made for publishing the numbers despite the sample size."
As for Ehrhoff, 94.0% of all shots while he was on the ice were at those first three strengths. I don't agree with the notion that "Ehrhoff got significant 4on4 TOI" - he got 16 times as much TOI at 5-on-5 as he did at 4-on-4. Let's look at the difference in Ehrhoff's Corsi percentage (=Shots For/[Shots For + Shots Against]) if we exclude and exclude 4-on-4 play:
Not only is the difference negligible, but it is also not true that "Ehrhoff...excelled at 4on4." At least not last season!
I am willing to accept that Christian Ehrhoff could be a particularly good player at 4-on-4. His 80 minutes at 4-on-4 last year were undistinguished, but, as I mentioned above, such a small sample size has virtually zero predictive value. And according to the poster, "[i]n Ehrhoff's case, he stands out as we ran the numbers separately." I'd love to see these separate numbers because I can't construct a compelling case in Ehrhoff's favor.
From the original poster -
"Ehrhoff had at least 2 goals at 4 on 4, one a game winner. I agree that the sample size is small, but it would be interesting to see what a total stats picture, all situations would show, if you take his situational number for the predominant situations and compare them to the overall number. I did not do that stat. My suspicion is that it was enough to bump his overall number. His overall shot accuracy went way up when taking his overall number and comparing it to the situational numbers that you have posted."
Well, as you can see in the table above, Ehrhoff's Corsi percentage was actually worse when we include 4-on-4 situations. That he scored two goals in 80 or so minutes of play at 4-on-4, including a game winner, is not indicative of future success. It's not necessarily an indication - given his lower Corsi percentage at 4-on-4 - that the sample of his play we saw even represented good performance.
"As an aside, he did get significant special situations TOI relative to other players on the team, 3on5 and 4on4. I would really like to see yourself or the NHL break out special situations because although they do not occur often, they are often the make or break situations in a game. 3on5, 5on3, 3on4 and 4on3 seem to have a lot of difference between teams."
We're really talking about two different things here: observed performance and true talent. Here is the observed performance of NHL teams at different man-advantages:
|Goals For/60||Worst||Best||% Variation||TOI|
In 3700 minutes of 5-on-5 time per team, the range of observed performance is close to that of true talent. But teams with low goal-scoring rates have typically underperformed, while teams with high goal-scoring rates have overperformed. The true talent range is much smaller, as evidenced by a 30% top-to-bottom variation in shot rates.
At 5-on-3, we see massive differences in observed performance, but does this match up to a correspondingly large range of true talents? Do we believe that there are teams that would continue to score zero goals at 5-on-3? No - using eight minutes of ice time is simply not an accurate indicator of a team's abilities. When we exclude 5-on-3 from our analysis, we introduce - in the absolute worst case - 5-7% uncertainty in our results, and virtually zero uncertainty into the predictive value of our tools.
Just to give you one example: the 2007-08 Nashville Predators were 26th in the league at 5-on-3. In 2008-09, they swapped Alexander Radulov and Marek Zidlicky off the PP and replaced them with Steve Sullivan (and extra minutes from all of their D.) They were 5th overall in 5-on-3 PP performance in 2008-09. Of course, their 5-on-4 performance was in the bottom 10 in the league both seasons, and got worse when they dumped Zidlicky, who is a solid PP defenseman. There's simply no evidence that a bad PP team dumped its best player, got worse at 5-on-4, but somehow became a much better team just at 5-on-3.