November 17, 2016

Baseball Therapy

Can a Closer be Worth $100 Million?

by Russell A. Carleton

the archives are now free. All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible. Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month Recurring subscription - cancel anytime. Subscribe for $39.95 for one year a 33% savings over the monthly price! Purchase a $39.95 gift subscription a 33% savings over the monthly price! Already a subscriber? Click here and use the blue login bar to log in.

If rumors are to be believed, Aroldis Chapman is looking for a nine-digit number on his next contract. Presumably, Chapman is looking for a four- or five-year deal, which would put his annual salary somewhere in the range of $20-25 million. In a market where teams also have the option of turning to Kenley Jansen or Mark Melancon, it seems a lot to ask, but hey, why not ask for a million for every mile per hour on your fastball. (I plan on demanding $40 million this offseason, if anyone out there is looking.) Maybe Jansen and Melancon are looking for similarly huge paydays.

It’s a lot to ask any four- or five-year contract to still be worth the big commitment at the end, but even with that aside, most “elite” closers usually only check in with WAR values around 2-3. The problem with relief pitchers is that while we tend to value free agents through the lens of WAR (get ready for plenty of “$8 million per WAR” reflex posts), we know that WAR doesn’t always do a great job of capturing the value that relievers bring to their teams.

WAR has the unfortunate habit of stripping the context out of everything, pretending that closers pitch in “normal” situations all the time. The entire point of a closer is that he’s the guy you bring into the game at the points of highest leverage. Whether he’s the best or the worst, he is specifically tasked with pitching at the times when the game hangs in the balance.

On top of that, we’re also entering a very strange place when it comes to dollars per WAR. At this point, the prevailing theory of $8 million per WAR is starting to look a little silly. The highest salaries in the game are just barely over $30 million, but the best players routinely crack the 8-WAR barrier. In theory, a 3-WAR player is worth $24 million, and a 4-WAR player should be the highest paid player in the league. So, we have a measure that often fails to accurately reflect the contributions of a closer and a salary thumbnail rule that seems to be broken. Great, how are we gonna estimate what an elite closer should be paid?

***

I think we need a slightly less nuanced view of what the closer’s job is. Over the course of 40-something save opportunities, a closer can add quite a few actual wins (forget all this abstract, context-neutral wins above replacement stuff) to his team’s ledger. He pitches in the ninth with a small lead. While it’s always shaky to try to give all of the credit (or blame) to the pitcher, if the closer does his job, his team goes home victorious. If he doesn’t, at best, the game is tied, and at worst, he gets to take a nice slow walk past the other guys who are throwing shaving cream pies at each other.

The job that the closer does is often literally the difference between a win and a loss that day. We could try to use Win Probability Added (WPA) to value a closer, but even that has problems. For example, there is more WPA to be had in the ninth inning of a one-run game than a three-run game, but again, the closer’s job is to protect whatever’s given to him, and it has a binary outcome as to whether or not he did his job. A bad closer can also drop a lot of WPA in those situations. Maybe the closer’s value is in what he prevents from ever happening.

Maybe we can find a way for a $100 million contract for a closer to make sense.

Warning! Gory Mathematical Details Ahead!

Well, for one, despite the fact that scoring is down, the late-inning comeback is alive and well, so having a shutdown closer is at least just as important as ever. There’s also no evidence that either save-worthy leads or even one-run leads in the ninth are becoming more (or less) prevalent. Here’s a graph showing both, as a percentage of games played, so there are just as many save opportunities floating around as usual.

There is actually a moderate amount of stability from year to year in the number of save situations that teams find themselves in. I used an AR(1) intraclass correlation over three years of data (2013-2015) to find out whether teams that were high in save opportunities in one year would be high in another year. (For those unfamiliar with the technique, think of it as a year-to-year correlation that is able to scan across multiple years.) The AR(1) rho was .523, which indicates moderate correlation from year to year.

Not surprisingly, teams that won more games overall had more small (1-3 run) leads to protect going into the ninth inning. The correlation between the number of save-worthy leads in the ninth and the total number of games won was .572. That’s a bit of a chicken-and-egg analysis, because if a team had a lot of save-worthy leads, and since most saves are converted, then of course, they’ll win a lot of games. But it makes sense that teams that are good at winning games are the kinds of teams that are already good at providing leads for the closer to nail down.

But then something interesting happened. While the number of save situations is correlated with the average number of runs given up by the team (r = -.505), the number of save situations a team finds itself in is not correlated with how many runs they score (r = -.024). These findings continued to hold when I included overall winning percentage in the model, as well as--to make sure I wasn’t pricing in the closer’s actual contributions--limiting runs allowed to reflect only a team’s performance in first eight innings of each game, and holding the league’s overall run environment constant.

So, the team we’re looking for that will get the most out of a closer is a team that is already very good, but good in a specific way, namely that they don’t give up a lot of runs to start with. The shape of their offense is actually largely irrelevant (within the limits of normal offense--if a team scored 25 runs per game, they wouldn’t have many one-run saves, but that team isn’t out there.) But even then, we need to think about how big an effect that is.

I ran a regression (data from 1996 to 2015) predicting the chances that a game would include a ninth-inning save for a team, based solely on their wins for that season. According to the regression, a team that won 90 games might expect 43.2 save opportunities over the course of a season, while a team that won 81 games might expect 40.3, so the playoff-caliber team is going to have three more save situations to give to the closer than the average one.

I then added in the team’s runs per game allowed to the regression model. I assumed that I was looking at two 90-win teams. I gave one team the best pitching in baseball in 2016 (the Cubs, 3.43 RA per game), and found that this team might expect 48.2 save opportunities over the course of 162 games. A 90-win team with league-average pitching (4.48 RA per game) would expect only 44.9 save opportunities.

Being on a winning/playoff team will net someone about three extra save opportunities over the course of a season, but being on the right kind of playoff team will net three more than that. So, we could hypothesize that there’s a team out there that would have six more save opportunities (give or take) than an “average” team. Having a good closer would be somewhat more important (and more valuable) to that team.

A closer having a good season nails down about 90 percent of save opportunities. A closer having a mediocre one hits about 80 percent of them. In theory, that gap of 10 percentage points is worth four extra games on an average team. And again, a blown save literally represents turning a win into--in descending order of goodness--a tie game, a deficit going into the bottom of the ninth, or an outright loss. If we value that around three-quarters of a win given away, a good closer is worth three extra wins (not wins above replacement, win wins) each year.

For the right kind of team, where they get six extra save opportunities, there’s another half-win of value in there. There’s a little bit of a premium that a playoff team should pay to get an elite reliever (although not “an order of magnitude” bigger). So, if we could make the case that Chapman (or Jansen or Melancon) were a good bet to nail down 90 percent of their save opportunities, like a good little closer should, then they should probably be paid commensurate with the fact that compared to a bad closer, they will likely add about three wins worth of value to the team. (Which is roughly what WAR usually says anyway for an elite closer.)

***

There are going to be some out there who ask “well, what about the playoffs?” Suppose that a team knows they will make the playoffs, and now is looking to win the World Series. They will (at most) be playing an extra 19 games, of which it’s likely that about a quarter of them will have a save opportunity. Maybe a little more because it’s the playoffs. Call it six or seven save opportunities. A good closer turns in a save an extra 10 percent of the time (0.7 extra saves), so perhaps one game over the whole postseason run that a team might have won that they lost because they didn’t spend the extra few million on that elite closer.

And that one game has to be the one that knocks them out of winning the World Series that they otherwise would have won. Yes, it most certainly could happen, no question, but don’t get caught up in the emotional aspect of it. If a team did blow a lead in a close game that did knock them out, everything would be blamed on the closer situation, but that’s bad probabilistic reasoning. There are a lot of ways to lose a postseason series and you have three chances to lose one. People aren’t counting on all the other ways that you could have been knocked out of the playoffs, even if you had gotten the elite closer who shut down that one game. You could lose the next three games by 14 runs, and it wouldn’t have made a bit of difference whether you had locked down the game before it.

Actual Retail Price ...

What’s the proper price for a three-win player at this point? If we still believe the whole “$7-8 million per win” line, then Chapman and probably his compatriots could make the case for at least a salary of $20-25 million. But if a team is going to spend $23 million, why lock up that much salary space on a deal that is mostly just market rate, when the compression at the top of the salary structure means that another free agent is probably coming soon who would actually be a better dollars-per-WAR value?

There’s bound to be a correction at some point, but in which direction? Will high-end talent begin to be paid $50 million per year, or will the market settle a bit and scale back the dollars-per-win figure so it’s a bit more linear? I’d suggest that just about any way you slice it, an elite closer is worth about three wins to a team, and so it puts him in that sweet spot where the market seems to make no sense. So, are Chapman, Jansen, or Melancon, actually worth $20-25 million per year? It’s possible that they’ll get it, but if they do, I think it says less about them and more about where the market for free agents is headed.

Russell A. Carleton is an author of Baseball Prospectus. Follow @pizzacutter4
Click here to see Russell's other articles. You can contact Russell by clicking here