Commenter Saskhab had some concerns about the assumption that OT performance is primarily luck. In particular: "I don’t think we can use 5 on 5 play as the predictor for 4 on 4 play."
I think, in the long-run, this is true. If a team played 10,000 minutes of sudden-death OT 5-on-5 and 10,000 minutes of sudden-death OT 4-on-4, I think we’d find that the 4-on-4 play was a much more reliable indicator of future OT performance. But as is often the case in sports, we don’t have large observed samples like that – what we have is a very small sample of the event in question (4-on-4 OT) and a larger sample of a dissimilar event (5-on-5 play in regulation.) To give you an example, the Montreal Canadiens have played a total of 60 minutes at 4-on-4 so far this season, compared to 1076 minutes at 5-on-5 – 60 minutes is not a lot of time to prove your abilities.
I looked at each team's shot differential in 2008-09 and divided shots into two groups: 1) 5-on-5 shots taken in tied games; 2) 4-on-4 shots taken in OT (when the score is also tied, obviously). I then compared each team's performance in odd and even games, and their performance in the first half of the season and the second half. Here are the results:
| 5-on-5, O/E | 5-on-5, 1/2 | 4-on-4, O/E | 4-on-4, 1/2 | |
| R^2 | 14.9% | 13.6% | 2.4% | 0% |
| Regression to the Mean | 61.5% | 57.6% | 84.5% | 100% |
Odd/Even games are abbreviated (O/E) and First/Second games are abbreviated (1/2). As you can see, 5-on-5 shot differential is a better predictor of future 4-on-4 performance – in fact, 4-on-4 shot differential in the first half of the season has no relationship with 2nd-half shot differential. This is primarily because teams played 25 times as much at 5-on-5 as they did at 4-on-4, which gives us a much better assessment of their true talent than the limited amount of time they spent at 4-on-4.
