In the discussion of the Washington Capitals goaltending situation, I see four main points of contention (excluding the torrents of insults):
1) There's a rejection of the idea that PK save % is luck-driven, and instead that it must be due to 'shot quality,' whether or not anyone has any evidence of it.
2) There's an assumption that we know nothing about a young goalie other than the limited performance we've observed in the NHL. This rejects the idea that goalies come from a population with a known distribution.
3) Health is not a skill - we need not consider Washington's third-string goaltender when trying to determine how good their goaltending is when their starters are injured.
4) Semantic issues: people don't think that noting that Washington reduced its PK shots against this season pays sufficient homage to the Capitals new PK system; and people don't like the idea that Semyon Varlamov, Michal Neuvirth and the eventual third-string starter combine to be "below-average," even though their performance is exactly that.
I think that's a faithful recording over a few hundred comments and I think I addressed all of these issues in pieces one, two and three on this subject. Frankly, I reject the notion that I haven't somehow addressed concerns about the analysis. There are a lot of wholly unreasonable expectations of me - while I have to provide evidence for every statement I make, this requirement doesn't run both ways. You can see a great example in my back-and-forth with Caps' writer Brian McNally, who writes:
“The new system…has limited PK shots by 2.9 fewer per 60 minutes of play than last season. That doesn’t help improve a goalie’s save percentage?”
Now there's an extensive body of work out there that shows how much team-level save and shooting percentage is luck-driven and that there's no relationship between shot volume and expected save percentage. I don't expect Brian to do a bunch of research for a blog post, but there's a clear double-standard here. If I wrote something like that that's wrong and provided no evidence, I'd never hear the end of it. I accept that I've imposed higher standards on myself for evidence than a reporter does. But in this case, I wrote something that was correct, based on copious evidence available at Objective NHL, and I was roundly attacked for it.
When a real torrent of homer fans show up at my site, people seem to expect me to analyze every idea that's thrown my way, even if there's no reason to think that they might be valid - zero evidence. And if I refuse? In addition to the barrage of insults that get thrown at me (hypocrite? seriously?), I'm branded closed-minded, even if I've spent countless hours studying the issue and have undergone a transformation in terms of the way I see the issue. Do you think 'shot quality' is major long-term driver of outcomes in the NHL? I once assumed that it was, but then Vic Ferrari, Objective NHL and Tom Awad all provided evidence to the contrary. I haven't seen anything that would change my mind back. Think a rookie with a high shooting percentage (or a goalie with a high save percentage) is likely to continue with it? I've made this mistake before, but I realized that I was wrong - this article is a pretty insightful look at what I was missing; you're welcome to write your own piece that shows that the author is wrong.
I'm not going to lie - if people expect me to immediately do the legwork to prove to the validity (or lack thereof) of their own claims to them, I'm not going to be particularly responsive. That doesn't reflect closed-mindedness - that reflects the reality that this site isn't my day job or even one of my main consulting gigs. I would hope that people who have ideas do what many people out there have already done - sit down with their computer and figure them out, whether it involves spreadsheets, crazy programming or watching a lot of game film. We all benefit from the knowledge that one additional person's efforts produce.