This appeared in Hockey Prospectus, June 1, 2009:
Consider three groups of goaltenders: what I'll call "Call-Ups", who played at least 10 games in the AHL in one season, followed by at least 10 games in the NHL the next season; "Shuttles", who played 10 games in each league in the same season; and "Sent-Downs", who played 10 games in the AHL the year after playing in the NHL. Their collective goaltending statistics are in the table below:
|Callups||AHL Prev Yr||2.68||0.912||2.72||0.908|
|Shuttles||AHL Same Yr||2.74||0.911||3.00||0.902|
|Sentdowns||AHL Next Yr||2.58||0.914||3.11||0.896|
If we only looked at their NHL statistics, it wouldn't be surprising that the "Call-Ups" perform better than the "Sent-Downs" in the NHL - after all, goalies get sent down when they do poorly, so the "Call-Ups" group should include a lot more goalies who played well and stayed in the NHL for the foreseeable future. However, don't we expect NHL performance to be a reliable indicator of future AHL performance? It is very surprising that a large group of players with such a wide range of NHL performance would play equivalently well in the AHL, even allowing for variations in player age.
We can tie ourselves in knots trying to come up with an explanation for why this happened, but I think the answer is simple. With an unproven goalie, teams can not tolerate a losing streak, so newly-arrived goaltenders have to play well immediately in the NHL. The difference between an NHL All-Star and a fringe goaltender is approximately one goal every other game. It doesn't matter that it might take 60 or 70 games to tell whether a goaltender has a true talent save percentage of .910 or .900 - if he plays poorly in his first 10 or 20 games, he gets sent down and somebody else gets a chance.
It should be obvious that this is not an optimal process for determining which goalie has the best skills: during the 2nd round of the playoffs this year, plenty of people were ready to write off Nikolai Khabibulin as a fringe NHL goalie after he lost Game 1 and Game 3. However, barring injury, he was still the guy who significantly outplayed his expected save percentage during the season. Obviously two bad games were not representative of his ability, but if he was a no-name goalie with a two-way contract, he'd be on the bench.
All of this is reflected in how difficult it is to scout and evaluate goaltending skill. A goal-scoring forward's career trajectory is pretty straight-forward - the earlier he plays in the NHL, the longer his career. To put some numbers on it, a 22-year-old rookie skater in the NHL plays an average of nearly 500 games, while a 26-year-old rookie expects 250 games. So there's certainly nothing predictable about goaltenders. They peak over a wide age range: a 22-year-old rookie goalie and a 26-year-old rookie goalie are both looking at a 275-game career, on average.
This is why we see goalies like Niklas Backstrom appear out of nowhere at age 28 and win a starting job in the NHL. It's also why we can't tell after three seasons if Backstrom is one of the best goaltenders in the NHL or simply a product of Minnesota's tough team defense. Again, the difference between the All-Star Game and a trip to Springfield is just one goal every other game - as long as teams are forced to use actual performance as an indicator of true talent, they'll never know whether they've got the right guy in goal.