Bayesian approach to analyzing goalies, Part 3: Updating estimates with more data

In Part 1 of this series, we gave an introduction to the series, and the motivation behind this new method of analyzing goalies.  In Part 2, we started with an example of Luongo's first 10 shots to build some intuition for what estimates this method gives and the advantages it has over traditional SV% or even strength save percentage (ESSV%).  We got a picture like this:

After 10 shots, Luongo's curve (black) is still very similar to what our prior expectation was based on typical ESSV% in the NHL. Since he saved 10 out of his first 10 shots, his curve (and the corresponding dot) is shifted slightly to the right. 

In this article, we'll continue this example and discuss the animation that we showed at the end of Part 2.  The animation shows what Luongo's curve looks like after 20 shots, 30 shots, 40 shots, etc.:

Luongo's distribution curve goes to the left, but quickly moves back to the right, and seems to stabilize pretty quickly.... the black dot seems to bounce around .930-ish area after the first 500 or 1000 shots.    As we add more and more shots, the curve gets narrower, and the peak of the curve gets higher. This indicates that as time goes on, and we get more and more information (i.e. we use more and more shots in the model), we are becoming more and more certain of our estimate of Luongo's true ESSV%.  In other words, our estimate is getting more and more precise. 

We added a blue dot to this animation which represents the observed ESSV%.  In the very beginning, it starts out off the screen to the right.  (Recall that in Part 2, we said that with 10 saves in his 10 shots, Luongo's ESSV% was only 1.000.)  Luongo allowed 3 goals in the first 7 shots of his second game, so the blue dot swings off the screen to the left.  The blue dot then swings back to the right and bounces around there for a while, way up past .940.  After a while, it stabilizes a bit, and is pretty close to the black dot.  This blue dot is kind of showing what we know about ESSV%: extreme values when the number of shots is small, very unstable in the beginning, but with a significant number of shots, it begins to stabilize. 

Another observation is that the black dot is always between the blue dot and the red dot.  This is true in the beginning when the blue dot is off the screen to the left, and when the blue dot crosses over to the right half of the figure, so does the black dot.  This is illustrating the "regression to the mean" that is going on.  Our Bayesian estimate for Luongo's true ESSV% is closer to the mean than his observed ESSV%, whether his ESSV% is higher or lower than the mean. 

During the beginning of the animation, the regression is pretty heavy.  In other words, the black dot is pulled way towards the red dot.  The prior distribution is dominating and has a big influence on our estimate.  This is probably preferred... we don't have much data, so our prior information has more influence on our estimate.  The blue dot, which is based solely on the limited data, is pretty erratic in the beginning with pretty extreme values.  

But as time goes on, the black dot moves further from the red dot and gets closer and closer to the blue dot.  The more data we get, the less influence the prior has, and the more influence the data has. This method is automagically regressing Luongo to the mean by an appropriate amount based on how much data there is.  

It might be helpful to look at a non-animated figure to emphasize what is going on.  Here's the position of the blue dot and black dot over time:

The horizontal axis is time in the animation.  More precisely, it is the number of shots used in the model.  The vertical axis is ESSV%.  So the curves are showing ESSV% after n shots, where n ranges from 10 to 5230.  The colors are the same as in the animation: the blue curve corresponds to the blue dot (observed ESSV%), the gray/black curve corresponds to our Bayesian estimate, and the red line is league average. 

The black appears to be pretty well-behaved and stabilizes quickly.  The blue is much more erratic in the beginning, with large swings, and it appears as a pretty noisy signal before stabilizing.  The black is always between the blue and the red.  In the beginning, the regression to the mean is heavy and the black is closer to the red.  After a while, when we have more data, the black is closer to the blue, and the two curves pretty much move in tandem. 

In Part 4, we'll talk about the regression to mean a little more, and start showing some results for the rest of the league's goalies.  We might even compare Luongo and Schneider, since that's never been done before.  This will be an example of a couple of other benefits of our Bayesian ESSV%:

1. it is useful when comparing goalies who have faced different numbers of shots. 
2. it is useful for answering questions like "what is the probability that Luongo's true ESSV% is greater than Schneider's?"

Links to other Parts:
Part 1 - Introduction
Part 2 - An example using only 10 shots
Part 3 - Updating estimates with more information
Part 4 - Regression to the mean, Luongo vs Schneider, Thomas vs Rask, Hiller vs Fasth