Bayesian approach to analyzing goalies, Part 4: Regression to the Mean, Luongo vs Schneider, Thomas vs Rask, Hiller vs Fasth

Last time, in Part 3 of this series, we saw what our Bayesian estimate for Luongo's true ESSV% looks like over time; i.e. when we get more and more information and use more and more data.  See here for links to all posts in this series. 


In this article, we'll show results for more goalies than just Luongo. I haven’t really heard much analysis of the goaltending situation in the Vancouver (joke). I’ve felt burdened by this obvious need that the hockey world has (joke), so I thought I’d start with a little statistical analysis of Roberto Luongo and Cory Schneider (not meant to be a joke, though you may disagree after reading this). Really, I just want to use them as an example to help illustrate this approach to analyzing goalies. 


One of the things we noticed in Part 3 is that there is some regression to the mean going on.  Luongo's Bayesian estimate was always between his observed ESSV% and the league average ESSV%.  Let's start with a figure that shows that this "regression to the mean" happens for all goalies. 
 

Bayesian approach to analyzing goalies, Part 3: Updating estimates with more data

In Part 1 of this series, we gave an introduction to the series, and the motivation behind this new method of analyzing goalies.  In Part 2, we started with an example of Luongo's first 10 shots to build some intuition for what estimates this method gives and the advantages it has over traditional SV% or even strength save percentage (ESSV%).  We got a picture like this:


After 10 shots, Luongo's curve (black) is still very similar to what our prior expectation was based on typical ESSV% in the NHL. Since he saved 10 out of his first 10 shots, his curve (and the corresponding dot) is shifted slightly to the right. 

In this article, we'll continue this example and discuss the animation that we showed at the end of Part 2.  The animation shows what Luongo's curve looks like after 20 shots, 30 shots, 40 shots, etc.:

Bayesian approach to analyzing goalies, Part 2: An example

In Part 1 of this series, we gave an introduction to this method of analyzing goalies and the motivation behind it.  We discussed why many analysts use even strength save percentage (ESSV%) instead of traditional SV%, and talked about two problems that ESSV% has which we'd like to deal with.  This series focuses on Problem 1: ESSV% tends to be very inconsistent especially for goalies who have faced a relatively small number of shots.  Our approach is a Bayesian analysis, where we can use "prior information".   

Let's begin with a simple example.  Let's consider the first 10 shots that Roberto Luongo faced during the 2008-09 season.  None of these shots were goals.  If this is the only information we have about Luongo, what should our estimate of his true ESSV% be?  In other words, what should we expect his ESSV% to be going forward?  We knew a lot about Luongo at the beginning of the 2008-09 season, but for the sake of explaining what's going on, let's pretend we didn't.

One approach to answering this question is to use his observed ESSV%. In this case, his ESSV% is 1.000 since he had 10 saves on these 10 shots.  This estimate is pretty high... 1.000 is way higher what any typical NHL goalie would ever have.  This is a pretty extreme example, since we are using only 10 shots, and it's sort of obvious that observed ESSV% should not be our estimate in this case.  But it will help us explain what is going on, and this kind of thing can still happen when using 100, 500, or 1000 shots, though the observed ESSV% won't typically be as extreme. 

Instead of using observed ESSV%, we could use a Bayesian estimate.  Suppose we know that the league's ESSV% are typically distributed like this:
 

Bayesian approach to analyzing goalies, Part 1


Save percentage (SV%) is a statistic that is commonly used to analyze goalies.  Analysts have started using even strength save percentage (ESSV%) instead, for various reasons. One reason is that there isn't much data for special teams, so it's hard to draw any conclusions about performance or ability on the penalty kill.

Another is that a goalie doesn't control how many special teams situations he has to face, and so doesn't control how many special teams shots he has to face.  If a goalie's team takes a lot of (non-coincidental) minor penalties, that goalie will face a lot of shots on the penalty kill.  Those shots are tougher to stop, and this will tend to drag down the goalie's SV%.  Even if he is one of the league's top goalies, his SV% may not reflect that because he is facing so many more short handed situations than the other top goalies in the league.  ESSV% avoids all this.

There are still a couple issues with ESSV% though. 

Problem 1: ESSV% still tends to be very inconsistent especially for goalies who have faced a relatively small number of shots. This has been noted by several analysts. The top three results in a google search for “save percentage regression to the mean hockey” are a good sample of articles that discuss these issues, and why ignoring these issues can lead an analyst to draw questionable conclusions about the ability of a goalie or about the other players on a goalie’s team.

More places to buy 2013-14 Hockey Prospectus


See here for links to various places to buy the book.  There are .pdf versions and paperback versions.


As noted at that link, the cheapest paperback is available from CreateSpace using the discount code 7B8F3CZ7.  It is $22.95 after discount. 

The .pdf version is $12.95. 


Hockey Prospectus 2013-14 now available

Hockey Prospectus 2013-14 is now available for purchase. See here for details on how to make said purchase. 

For those interested, I contributed to the BUF, OTT, and WPG pages.  For WPG, I wrote about stuff related to realignment, and how WPG's travel will be affected.  For BUF and OTT, I analyzed their goalie situations using a new technique. 

I'll write more about those techniques in a forthcoming post, but the basic idea is that the approach automagically regresses a goalies estimated true Sv% towards the league average, especially for those goalies who have faced relatively few shots. 
 

PLAY - More analysis and top 10 PLAYers

In Parts 1 thru 3, we developed a new playmaking metric that is (1) more consistent than assists and (2) better than assists at predicting future assists. had an observation about the results that we'll study a little further in this article.  Along the way, we'll give the top 10 PLAYers (capitalization and pun intended) from 2009-10, and how they did in 2010-11.


PLAY: A Playmaking Metric, Part 3 - Results


In Part 2 of this series on a new playmaking metric PLAY, we talked about "altruistic contribution" based on shots.  It is basically the difference in shots taken by a player’s teammates (excluding the player’s own shots) when he is on the ice versus off the ice.  It’s kind of like a shot-based WOWY that doesn’t include the player's own shots.  If a player's teammates take more shots when he is on the ice versus off the ice, his altruistic contribution will be high.

Now we'll use this altruistic contribution to build our playmaking metric PLAY, and show that PLAY is better than assists in two quantifiable ways: (1) it is more consistent than assists, and (2) it is better than assists at predicting future assists.  

PLAY: A playmaking metric, Part 2 - Altruistic Contribution

Part 1 of this series on a new playmaking metric PLAY, we gave an overview, and discussed marginal contribution, which is basically some form of WOWY.  See here for Part 1 and here for the full paper.  In this part, we'll discuss competitive and altruistic contributions.  Altruistic contribution is what we'll use to compute PLAY.

PLAY: A Playmaking Metric, Part 1 – Intro



I wanted to share a recent project I did with friends and colleagues Chris Weld and Chris Arney titled "Quantifying playmaking ability in hockey".  The one sentence summary is that we developed a metric for quantifying playmaking ability in hockey that is better than assists in two quantifiable ways: (1) it is more consistent than assists, and (2) it is better than assists at predicting future assists.  

In this article, we'll give an overview of what we're doing, and the motivation behind it. 

Trade deadline article at ESPN.com

Ryan Wagman (@RAWagman) and I recently wrote an ESPN.com article on trade deadline fixes for each of the Atlantic Division teams.  Check it out here if you have ESPN Insider, or here if you are a member at HockeyProspectus.com.  For the other divisions, there are links on the right hand side of the ESPN article.

Disclaimer: we finished the article before the Pens' flurry of trades.  We picked Adrian Aucoin for the Pens, suggesting the Pens biggest need was a depth defenseman.  Since then the Pens acquired a different defenseman, Doug Murray, so that sort of coincides.  Of course, the Pens did some other stuff too. 

Wired.com article on realignment

Samuel Arbesman (@arbesman) wrote a nice article at Wired.com titled Algorithmically Realigning Sports Leagues. The article is a nice summary of the realignment project that Bill Pulleyblank and I worked on. Check it out here, and the rest of our realignment stuff here

Analysis of NHL's realignment plan

I just finished up an article over at HockeyProspectus.com, analyzing the NHL's new realignment plan

At first glance, the new plan might appear bad for the teams in the northeast and the Florida teams, who are all in a division together.  But we show, among other things, that the new plan has a fairly small impact on their travel... just a small increase over what their travel would have been in the the "best" 4-conference solution (if by "best" we mean the solution that would minimize travel).

Links to all of the realignment articles are on the realignment page

 

More realignment figures

Just wanted to post some figures that are related to the discussion going on over at Tom Tango's blog about realignment.  I'd encourage you to read the discussion there first.  (I encourage you to read the rest of his blog regularly also.)

NHL Realignment

I recently finished a paper on realignment with friend and colleague Bill Pulleyblank.  The purpose of the project was to determine solutions to realignment that minimize travel for the league. We also compare our best solutions to the NHL 4-conference proposal from around a year ago. 

I also wrote a couple of articles about it on Hockey Prospectus.  Links to everything, plus some animations that are discussed in the second Hockey Prospectus article, can be found on the Realignment page.

Realignment Animations

I'll be writing some articles on realignment shortly.  In the mean time, here's a preview... I posted some animations related to this project on the Realignment page, including the top 100 solutions for each of the 4 major sports leagues.

Details about our method of generating and ranking these realignments is coming soon...