Commenter Saskhab had some concerns about the assumption that OT performance is primarily luck. In particular: "I don’t think we can use 5 on 5 play as the predictor for 4 on 4 play."
I think, in the long-run, this is true. If a team played 10,000 minutes of sudden-death OT 5-on-5 and 10,000 minutes of sudden-death OT 4-on-4, I think we'd find that the 4-on-4 play was a much more reliable indicator of future OT performance. But as is often the case in sports, we don't have large observed samples like that - what we have is a very small sample of the event in question (4-on-4 OT) and a larger sample of a dissimilar event (5-on-5 play in regulation.) To give you an example, the Montreal Canadiens have played a total of 60 minutes at 4-on-4 so far this season, compared to 1076 minutes at 5-on-5 - 60 minutes is not a lot of time to prove your abilities.
I looked at each team's shot differential in 2008-09 and divided shots into two groups: 1) 5-on-5 shots taken in tied games; 2) 4-on-4 shots taken in OT (when the score is also tied, obviously). I then compared each team's performance in odd and even games, and their performance in the first half of the season and the second half. Here are the results:
|5-on-5, O/E||5-on-5, 1/2||4-on-4, O/E||4-on-4, 1/2|
|Regression to the Mean||61.5%||57.6%||84.5%||100%|
Odd/Even games are abbreviated (O/E) and First/Second games are abbreviated (1/2). As you can see, 5-on-5 shot differential is a better predictor of future 4-on-4 performance - in fact, 4-on-4 shot differential in the first half of the season has no relationship with 2nd-half shot differential. This is primarily because teams played 25 times as much at 5-on-5 as they did at 4-on-4, which gives us a much better assessment of their true talent than the limited amount of time they spent at 4-on-4.