The League of Extraordinary Statisticians (LOES) is a weekly forum bringing together the top analytical minds in the hockey world to answer a variety of questions that straddle the line between stats analysis and something you might hear floating around section 304. They have agreed to answer these questions in a few paragraphs or less, and with minimal formulae. Because this is a forum, we'd encourage you to use the comments section to answer the questions yourselves, or to discuss or debate the answers given.
The LOES is not meant to represent the entire of the hockey stats community. There are a number of people that either were too busy or too difficult to contact for the purposes of the forum.
Goaltenders hold a curious place in our conception of hockey talent; they stand alone because it comes with the position, and in the process they carry a certain amount of blame for goals-allowed and none of the credit for goals-for. Once again, it comes with the position. For such a solitary existence, the goaltender is still greatly reliant on the performance of his or her team, so to a degree the stats community has been careful about praising goaltenders for accomplishments in such categories as goals-against average (GAA) or wins.
More after the jump.
Despite the fact that goaltending talent is closely tied to team talent, there is still a notion that there are goalies that stand above the rest, or "elite goaltenders." We can throw around the terms "good goalies" and "bad goalies", and they both are pretty vague ideas of talent and/or performance, but "elite" suggests truly recognizing a player above the rest. So, with that in mind...
How do you define an "elite goaltender" in today's NHL? Or, are there elite goaltenders in today's NHL?
Goaltending performance can vary greatly from season to season. A case in point is Ilya Bryzgalov, whose last four seasons' save percentages have been .907, .920, .906 and .920. I don't think you can be considered "elite" if you're only elite in odd or even years. Not that I'm sure what the "elite" label means...other than perhaps defining a netminder that's worth throwing a relatively big-money, long-term contract at. And there are a few such talents out there, regardless of the meager contracts we saw this past offseason.
How about a quick-and-dirty definition as a goaltender who's posted a top-20 save percentage over the past three seasons, or in four out of the last five seasons (including the past season)? Who would qualify?
The best of the best recently have been Vokoun, Lundqvist, Brodeur and Luongo, the only four goaltenders to post top-20 save percentages every season since the lockout. In addition, our definition would include Ryan Miller and Tim Thomas...and would you believe Chris Mason? On the cusp are Anderson, Backstrom, Bryzgalov, Halak, Hiller, Price and Ward, who would qualify if they made the top 20 this season. Notable netminders not making the cut are Fleury, Kiprusoff, Turco and the recently-departed Evgeni Nabokov. And young netminders like Howard and Rask need a longer track record to prove themselves as worth or not.
- Timo Seppa, Hockey Prospectus
I have written several posts on the topic of elite goaltenders. Specifically I am arguing that the teams that have recently won the Stanley Cup are not as elite as most historical Stanley Cup champions because they do not have elite goaltenders and that is a significant portion of having an elite team. The last three Stanley Cup winners had Chris Osgood, Marc-Andre Fleury and Antti Niemi in goal and none of them are elite. This is a sign that the NHL system with a salary cap and liberalized free agency prevents elite teams from being formed.
Elite goalies are goaltenders who are consistently among the top goalies in the league and expected to remain that way into the next few years. Typically there are about five of them in the NHL at any given time. If you or I or anyone else who follows hockey carefully had to write down a list of the top goalies in the NHL, we would likely have similar lists.
In the off-season, I ranked the top 50 players in the NHL. The goaltenders were Ilya Bryzgalov, Henrik Lundqvist, Roberto Luongo, Ryan Miller and Tomas Vokoun and you could make a reasonable argument for a sixth member of the group in Miikka Kiprusoff. I think Martin Brodeur's best days are over and he no longer has a spot on that list.
As the season is played this year, we may see other goalies put in strong enough performances that propel them onto this list and some of the current members failing. Jaroslav Halak, Tuukka Rask and Jimmy Howard are young goaltenders who have the best chances to move up. Tim Thomas is off to an incredible start, so I feel the need to mention him, although he is no longer young. I expect that Rask and Thomas cancel each other out because neither will likely play enough to be elite, but I could easily be wrong.
An elite goalie goes a long way toward building an elite team. In today's league where elite teams no longer exist, it is possible to cut the corner on goaltending or other positions and still win a Stanley Cup. Recently, goaltending has been the corner most frequently cut, but it is quite likely we will see others cut in the near future.
- Greg Ballentine, The Puck Stops Here at Kukla's Korner
The most effective way to define an "elite goaltender" is to look at save percentage - after adjusting it for situation (5-on-5, 5-on-4, etc - some just look at even-strength save percentage only). Gather it over an extended period of time, because one season is often too small a sample size. Then based on those two factors (sample size and save percentage) you can establish the goalie's true skill level with a confidence interval.
An elite goalie would be any goalie who doesn't overlap significantly with at least half the other goalies. As such, there are those that believe that Hasek was the last elite goalie; the last goalie that was consistently, provably and appreciably better than the average. That wouldn't be an unreasonable position to take.
I didn't talk about shot quality, but as soon as the NHL has improved the consistency of where shots are taken from we can use that to further adjust the situation-adjusted save percentage.
- Rob Vollman, Hockey Prospectus
Not really. I think Hasek is probably the only guy in recent memory who has truly stood out above the rest of the league. We break down the data in a few different ways and get different answers to the question of who the best goalie is. You could make an argument for ten different guys, probably. And the gap between them and others is not that large, plus you have guys like Jonas Hiller just floating around, waiting to be dominant in the NHL.
Well I’d say there are elite goaltenders, as by the definition of the word "elite" means the best and I’d say there is one goalie out there that is better than the rest even if by a slim margin. Whether or not we are good at identifying this elite goalie is another question. Based on the studies into shot quality and its relative insignificance in the long-run, one would say save percentage is a good indicator of success. Tom did do a study on team SQ though that concluded,
"For outlier teams that excel or are terrible at [shot quality], it is important to take into account. For most teams, it can be neglected."
Goalies more so than any other position require a substantial sample size of save data before we can say if the high performance level is for real or not. The only goalie in the NHL recently to consistently maintain a .920+ SV% has been Vokoun. Players like Stamkos can emerge in one season and we can see some, even if not full, evidence that he will likely be a good player in the future. For goalies we cannot make those conclusions in timeframes like that, so usually when trying to ask questions about goalies' true talent level based on one or two years worth of data you'll end up with the response "I don’t know."
- Corey Pronman, Hockey Prospectus
One can define things however one wishes, but the reader will almost always ignore the definition of common words like elite, and instead apply his own definition.
As with a few of the other categories of players and performance, "elite" proves to be a bit of a puzzle that has to be put together. What are we using for our analysis of goaltenders? What level needs to be achieved? Is there difference across eras that needs to be explored? How might a team's performance affect a goaltender's results? As you can see, there's no easy answer, though we do have a fuzzy picture of "elite goaltenders" that comes out of this.
Tomas Vokoun, Roberto Luongo, and Henrik Lundqvist get "maybe's" when it comes to being elite, but there seems to be serious questions whether today's cohort of NHL goaltenders are truly elite. I highly doubt any of the statisticians would question Dominik Hasek's place in the elite category, but to equate Hasek's level of play to performance in the contemporary era is debatable.
So, a few things to think over: are there elite goaltenders today? Or is the talent close enough from top-to-bottom to render the category inapplicable in this era? Should "elite" be a category constructed out of comps within an era (as Ballentine and Seppa seem to suggest), or should it be reserved for a handful of goaltenders throughout the NHL's history? When are we going to come to our senses and realize that Peter Ing was an elite talent that just routinely played cat-and-mouse with quantification?