Holistic metrics are a tricky bunch. We have the original, +/-, which has always had the benefit of being the “elder statesman” (don’t do it, Ben, don’t do it…elder stats-man?…ugh, I’m sorry about that) and the fact that it was a tool used prominently by the Montreal Canadiens. Recent efforts by Iain Fyffe, Rob Vollman, and Brian MacDonald have sought to tease out some of its shortcomings, with a certain level of success. The Corsi Number has been around for quite awhile, too, but has been embattled by shot-quality proponents and shot-recording bias, despite some smashing refinements of the numbers themselves. Alan Ryder’s Player Contribution, built in-part on the Bill James Win Shares model, is probably the only one of the bunch to elevate defensive players onto a near-equal plane with offensive players, but not everyone agrees that goaltenders should receive as much value as Ryder attributes to them. Tom Awad’s GVT has gained a lot of currency, in no small part because he’s made his methods and league data easily accessible to everyone. GVT still favors offensive player types (and is pretty favorable to goalies, too), but like Player Contribution it credits two-way players above all. Tom’s most recent effort, Delta and Delta SOT, is very promising but also brand-spanking new, so it will be interesting to see if it gets some more love or critique over the next couple of years.
A little while ago, Hockey-Reference.com came out with a new, holistic player metric, called "Point Shares," which attempts to use James' Win Shares in a more simplistic way than Ryder's approach. The methodology isn't too complicated; explained here, it boils down to dividing the contribution that each position makes to a win (somewhat arbitrarily), then essentially using a +/- like approach to determing that player's role in their team's points towards the standings. The players that benefit from this metric either are on good teams, are involved in a lot of their team's goals, both of those two, or in some cases are a goaltender on a mediocre team. Sounds pretty good, right?
Well, point shares do not appear to tell us anything that the previous metrics haven't. My initial feeling was that Alan Ryder has already worked with this approach, has seen how the numbers work out, and made numerous adjustments to improve it, so I'm really not sure that starting back at square one is going to help. Truth be told, its developer, Justin Kubatko, comes from a statistics background mainly in basketball (he founded Basketball-Reference.com, which later joined with NFL- and MLB-equivalent sites under the umbrella of Sports Reference LLC), and had previously applied his approach to the NBA before converting it for use in the NHL. At least from the description, it appears he's familiar with the work of Fyffe, Awad, and Ryder, but the explanation was he didn't have access to large chunks of the data and wanted to developed something for Hockey-Reference.com that they could just record and keep on there in a more open-source way. I don't know much of the backstory, whether he contacted these guys or sought cooperation or not, but I can tell you from experience that whenever I've had a question about anything they've done Iain, Tom, and Alan have been very accessible.
Anyway, point shares essentially does the same thing that +/- did (in the past): favors players on the best teams, in particular those who play big minutes. I guess the latter isn’t so bad, but the former doesn’t tell us much of anything. Take a look at Michal Rozsival‘s career numbers and guess which year he led all NHL defensemen in point shares…do the same for Derian Hatcher and two-time leader Brian Engblom. As for goaltenders, it’s very reliant on wins. In the pipes for a lot of your team’s wins? Congrats, Sean Burke, you are the best goaltender in 2000-01. Ditto, Murray Bannerman in 1983-84. I mean, these are intriguing results to a degree, but at the same time it seems very team-reliant, something that guys like Fyffe, Vollman, Awad, and Ryder (and Gabe, for that matter) have spent many years trying to weed out.
I will say this: in general, point shares and talking about these other metrics makes me think about holistic metrics in general. I mean, what do we want them to do for us? Tell us how big of a piece of the pie a player was, how their talent compares to the rest of the league, or both? Should they lump defensemen, forwards, and goaltenders into one group, or should they delineate their separateness? Should the divisions of players go beyond that? Should they be solely teleological (based on what happened, versus what could have happened)? Should they be "accessible" to the average fan (in other words, the methodology easily understandable)?
Some suggestions:
- Let’s keep/start working on making prediction models. If advanced stats want to gain currency, we need to go out on a limb and prove ourselves to be the “better experts.” HP took a big step in that direction by coming out with an NHL season preview, Gabe and others have done previews of a few teams…the next big step is rehashing those predictions, seeing which ones failed, which ones succeeded, and most importantly why.
- We should develop ranges based on performance data. Yes, X player scored this many goals, this many assists…but based on historical performance, with his shot totals and team shooting, what was the range of potential boxcars for this player? We are starting to get sufficient data to do this, and it can help with the above prediction models.
- Understand that the salary cap era has created another essential component of player performance: how much that player is producing/contributing versus what they’re getting paid. Player contracts today are constantly under the microscope. In my opinion, two of the more important new holistic metrics (possibly until Tom’s Delta is vetted) are Rob Vollman’s GVS and Alan Ryder’s PCValue (see page 4). Rob is using GVT, so credit to Tom is due, and obviously Alan is working off PC, but regardless this is another important component of analysis for the current era. Who is doing the most for less? Who’s worth X dollars? Who’s not? Are certain positions worth more than others? These are incredibly important questions.
As you can see, the opportunity to critique point shares has opened a whole can of worms, but in a good way. As we close another season of well-recorded data, we are getting to the point where we can achieve more and more with our metrics. Cooperative efforts, including the scoring chance recording that has occurred this year, interactions on hockey statistics-focused blogs, and providing informed critiques are only going to help. Okay, I'll stop now.
