In this article, we'll show results for more goalies than just Luongo. I haven’t really heard much analysis of the goaltending situation in the Vancouver (joke). I’ve felt burdened by this obvious need that the hockey world has (joke), so I thought I’d start with a little statistical analysis of Roberto Luongo and Cory Schneider (not meant to be a joke, though you may disagree after reading this). Really, I just want to use them as an example to help illustrate this approach to analyzing goalies.
One of the things we noticed in Part 3 is that there is some regression to the mean going on. Luongo's Bayesian estimate was always between his observed ESSV% and the league average ESSV%. Let's start with a figure that shows that this "regression to the mean" happens for all goalies.
In this figure, each gray dot corresponds to a goalie. The horizontal position of the gray dot is based on the goalie's observed ESSV%, and the vertical position of the gray dot corresponds to our estimate of their true ESSV%. The blue dots correspond to observed ESSV% and the red line is league average.
The key observation is that for each goalie, their gray dot is between their blue dot and the red line. This means for every goalie in the league, our estimate is between their observed ESSV% and the league average. Above average goalies (right side) are automagically pulled down towards league average, and below average goalies (left side) are automagically pulled up towards league average.
Notice that some goalies are closer to the league average than others. We have sized the dots by shots faced. Notice that the small dots are typically closer to league average. We saw this before in Part 3. When the number of shots is small, our estimate tends to be close to league average. When the number of shots is large, our estimate tends to be closer to observed ESSV%. The extent to which our estimate is pulled towards the mean is automagically determined by the model.
One thing we didn't previously stress is that this method is pretty useful for comparing goalies who have faced different quantities of shots. It is most useful when those goalies played on the same team so that we can assume they faced roughly the same quality of shots. We'll adjust for quality of shots in a future series, which will be better for comparing goalies on different teams.
As an example, let's start with Luongo and Schneider, who both played for VAN during the seasons we used for this series (2008-09 thru 2012-13). Luongo and Schneider had roughly the same even strength save percentage (.933, and .931) during the seasons that we are using. Since Luongo has faced about 3000 more shots than Schneider, we are more sure of Luongo’s ability. But how sure are we?
Let's look at the same figure as above, with Luongo and Schneider highlighted:
Let's look at the curves for Luongo and Schneider as well:
TrueESSV% Err ESSV% Shots
Roberto Luongo .930 .003 .933 5239
Cory Schneider .926 .004 .931 1932
The Err column indicates that the error bound for Luongo's estimate is smaller than that of Schneider, as we would expect.
In this case of Luongo and Schneider, though the gap between the two goalies changes, the order of the two goalies stays the same: Luongo has both a higher observed ESSV% and a higher Bayesian ESSV%. In other words, using both methods we would conclude that Luongo is the better goalie. But as we saw in the OTT and BUF write-ups in the 2013-14 Hockey Prospectus book, this isn't always the case. Often, the order of your goalies will switch. Let's give an example here, and highlight Tim Thomas and Tuukka Rask:
TrueESSV% Err ESSV% Shots Tim Thomas .9303 .0032 .9346 4723 Tuukka Rask .9294 .0038 .9356 2824
The gap between them isn't huge in either case, but the order did switch. See the OTT and BUF team pages in the 2013-14 Hockey Prospectus book for more extreme examples.
What about the goalie situation in ANA? Here are the results for Hiller and Fasth:
TrueESSV% Err ESSV% Shots Jonas Hiller .9259 .0030 .9278 5234 Viktor Fasth .9226 .0053 .9272 508
Hiller and Fasth have almost the same ESSV%, with a slight edge to Hiller. But Fasth has far fewer shots, so our estimate of Fasth's true ESSV% is pretty close to the league average. The gap between Hiller and Fasth according to true ESSV% (.0033) is larger than the gap in observed ESSV% (.0004). Also, Hiller's estimate is more precise (smaller Err). This analysis suggests that ANA shouldn't overreact to Fasth's strong performance last season and do something like trade Hiller. (Well, that's the conclusion if we are focusing only on performance and are ignoring contract status and cap hit.)
These kinds of conclusions are not new to the analytics community. You can eyeball their ESSV% and notice they are almost the same, you can notice the huge difference in the number of shots that these two goalies have faced (5000 vs 500). Analysts familiar with the idea of "regression to the mean" would expect Fasth's ESSV% to regress.
But what is new is that we now have a way to quantify these qualitative ideas. We also have error bounds on our estimates. We'll continue with examples like this next time. We'll also use our results to answer the question "What is the probability that Goalie A has a higher true ESSV% than Goalie B?"