December 22, 2010

Checking the Numbers

Relievers and the Value of Perfect Information

by Eric Seidman

Printer-friendly

Contact Author

One of the most interesting aspects of baseball transactions is that every move carries a certain level of risk. Without perfect information akin to an extreme level of accurate prognostication, teams can never be 100 percent certain how a player will perform. The best investment is a player who will produce at a very high level, but who comes as close to guaranteeing that high level of production as possible. Albert Pujols personifies this idea, as a bad season for him still involves above-average defense and a TAv north of .310. I will say right now, with as much confidence as I have in my mind and body, that Pujols will hit, at worst, .310/.370/.530 next season. He has proven himself capable of production far beyond that slash line and is as close to a sure thing as we have.

Signing Jeff Francoeur to play every day would be an example of the opposite effect, as there is ample evidence that he cannot handle such a role. We are virtually certain he cannot handle that type of role in a productive manner, so the risk associated with signing him extends beyond the normal types inherent in signing a player because of the increased level of certainty that he will not live up to expectations. The majority of players traded for or signed falls into the area in between both of these extremes, which makes evaluating transactions quite the tall, but fun, task. This is before even getting into the idea of evaluating moves on their merits at the time as opposed to retrospectively, which is a topic for another day.

Relief pitchers, however, muddy the waters, as they are as fickle in performance as teenage girls are with their crushes. In spite of this, teams often place a high premium on their services. Too often is a reliever signed to a multi-year deal off of 55 solid innings, only to prove that his success was a direct result of small sample size trickery. In other cases, the same hypothetical reliever might pitch well in the year following his signing, while his production erodes over the remainder of the deal. If signing a player who has proven himself unfit for an everyday role represents a terrible investment, signing one to a lucrative contract based on numbers derived in minute samples is not too far off.

When Scott Downs signed a three-year deal worth $15 million with the Angels, I got to thinking about how relievers signed to multi-year deals typically perform. I could at least somewhat understand it in Downs’ case, because he has performed very well for an extended period of time. When Jesse Crain and Matt Guerrier signed three-year deals, my head began to spin and I started wondering about how relievers perform over the lives of their multi-year deals. Do they live up to expectations? Does their production match, or exceed, the value of the contract? Are they signed to long-term deals so the team can reap the benefits of that first season through an added push in playoff contention?

On a more basic level, does a reliever signed to a three-year deal perform better than three relievers signed to one-year deals over the same time span? And are the relievers inked to one-year contracts given less money, on average, than those signing multi-year deals? If the answers to the last two questions are no and yes, respectively, then it would behoove teams to avoid doling out significant contracts to relievers. These are important questions to research because some teams have allocated the majority of their off-season spending to shoring up a bullpen (cough, 2007 Orioles, cough). If taking such a course of action is suboptimal in both the short- and long-term, then teams should spend their money elsewhere. However, my goal here is not to definitively answer these questions, but rather to open the discussion and explore why conducting studies along these lines are not as clear cut as they may seem.

Downs seems to be in the elite reliever mold, as his ERAs since 2007 have been 2.17, 1.78, 3.09, and 2.64. In a grand total of 236 2/3 innings over the same four-year span he has produced 8.8 WARP, a very good mark for such a small sample of production. But is he worth committing three years and $15 million to? There are a few ways to attack a question like this, but I am not interested in projecting his performance and figuring out if the translation to financial value exceeds the dollars paid out in the contract. Instead, I am looking to see how multi-year deals for relievers generally work out relative to their one-year counterparts.

To that end, I used the ESPN Free Agent Tracker and went back to the 2006 offseason, binning relievers based on whether they signed multi-year deals or deals for just one season. Multi-year relievers were recorded for the 2006 and 2007 offseasons, while I added all one-year relievers from the 2006-09 offseasons to my spreadsheet. For the multi-year relievers I recorded their WARP totals in each of the two years preceding their contract year, as well as over the life of the deal. The same process was repeated for the one-year wonders, though the lives of their deals were just one season.

We could repeat this process with WXRL, FRA, SIERA, or even ERA, but I will stick with WARP for now, as the more important idea is how to actually begin a study like this—too many articles have attempted to answer the same question, either starting with the wrong question or failing to use a control group, or even neglecting to include very important variables discussed later on. The table below shows the pertinent information for the multi-year relievers signed prior to the 2007 season, with the PY and Y columns representing WARP totals in the second preceding year, the prior year, and each of the years in the new contract. Note, too, that I excluded Mariano Rivera from the tables as he is a curve-buster.

Summarizing the table above, the pitchers averaged 37.3 years of age and signed deals to an average of 2.6 years and $12.4 million. In the second preceding year they averaged 1.1 WARP; in the preceding year they averaged 1.4 WARP. In the first year of their shiny new contracts, they reverted to the level of production two years before signing. And during the second and third years of their deals this group performed worse than any of the other years tabled above. The sample is small, of course, so nothing definitive can be drawn from this table. But this is a starting point.

The relievers appeared to be worth approximately $5-6 million leading into the deal and then saw their production plummet. Over the lives of the various deals, the multi-year relievers provided, on average, 2.3 wins. How did their one-year deal counterparts fare over the same time span? Well, there are far too many relievers to list in a table, so I will instead just summarize the information in the table below:

What is striking at first is how the ages are so similar across both groups. It isn’t as if the younger relievers are signing multi-year deals and the elderly are stuck going to year-to-year with teams. Additionally, the salaries go hand-in-hand with the deals, as a reliever is not going to sign a three-year deal for $5 million, just like it is somewhat rare to see a one-year deal worth $9 million or more for a reliever. The latter situation happens, but not all that often, while the former virtually never occurs in this day and age. Translating dollars to wins it appears that those signing multi-year deals during the 2006 and 2007 offseasons cost approximately $5.6 million/win in salary. The relievers signing one-year deals cost approximately $3.3 million/win in salary. And as the numbers above suggest, the relievers signed to one-year deals were not that far off of the multi-year relievers during their shorter contracts.

Given that, on average, a win is worth somewhere around $5 million per the current incarnation of MORP, multi-year relievers were slightly overpaid, while relievers signing one-year contracts were veritable bargains. However, and this is a big however, the data above is strictly salary-based. It does not include free-agent compensation for Type-A or Type-B free agents. The cost of signing a Type-A free agent is not merely the actual salary paid to the player, but that contract plus the value of the draft picks surrendered. According to this article by Matt Swartz, the cost of surrendering a first-round pick is $8 million, and the cost of giving up a second-round pick is $5 million. These values are derived from the historical value provided by players selected with those picks.

When a team signs a Type-A free agent they surrender a first-round pick, unless they finish in the bottom half of the sport, in which case they surrender a second-round pick. This is, of course, assuming that the lame-duck team offers the player arbitration. So if we have a pitcher who was offered arbitration, turned it down, was classified as a Type-A free agent, and signed a 3-year/$19 million deal with a team finishing in the upper half of the league, the new team actually spent $27 million when the cost of a surrendered first-round pick is factored into the equation. Here it becomes much more difficult to fully recoup the value of the contract with production on the field. In cases like this, teams may look to keep the reliever in the fold for an extra year or two as a means of trying to recoup more of the overall value of the contract and the compensatory picks.

If the current free-agent system receives a much-needed makeover and the compensation structure was updated, relievers would arguably benefit the most. Teams could sign relievers to one-year deals without having to give up a first-round pick, and they would not feel the need to add extra years to the deal just to potentially make up for the loss of the pick. Given that relievers are so fickle in their performance, doesn’t it make sense that the vast majority of them would not be offered multi-year deals if teams didn’t have to give anything extra up to bring them in?

There is a lot more to look into with a topic like this, but it is very important to understand that determining whether relievers are worth multi-year deals is nowhere near as easy as it might seem. Because of the structure of the free-agent compensation system, there is often a greater cost associated with signing relievers to long-term deals. Given their minute sample sizes of production it often becomes difficult to recoup the full value of the deal, which includes the value of any picks surrendered. Some teams like to make big splashes with their bullpens in the offseason, while others search the scrap heap for America’s Next Great Reliever.

Either way, it would be wise to constantly evaluate how these deals work out from an overall point of view, taking all of these factors into account, because if a strategy does not work it makes no sense to stick with it. With such a small sample it is impossible to concretely determine if multi-year relievers are poor investments, because with a larger pool of relievers the numbers may sway in the positive direction. Regardless, there are several variables that must be accounted for to even come close to accurately answering the question, and hopefully this is a good starting point. 

Eric Seidman is an author of Baseball Prospectus. 
Click here to see Eric's other articles. You can contact Eric by clicking here