August 30, 2006

Lies, Damned Lies

To Close Or Not To Close

by Nate Silver

My recent chat featured nearly a dozen questions about my comment in the last edition of LDL that Jonathan Papelbon "absolutely should not be moved from the closer's role" for the foreseeable future. Let me try and boil this down into the simplest possible terms:

1. We know that Papelbon is a great closer. In fact, he is having one of the five greatest seasons by a modern-day closer in baseball history.
2. A great closer is as valuable as all but the very best starters, once we properly account for the effects of leverage. Papelbon's WXRL this year is 6.6, which is higher than that of any starter not named Johan Santana (6.7). (Note that we call this statistic SNLVAR for starters. But it measures the same thing--wins added above replacement--and is measured in essentially the same way.)
3. It is not certain at all that Papelbon would be comparatively as effective if he moved to the starting rotation.

Ignoring some esoteric arguments about whether being able to pitch in high-leverage situations is a skill, the first two points are more or less indisputable. It's the third point where the debate lies.

It would help if we framed the discussion a little bit more precisely. Assume that, going forward, Papelbon would provide a 2.00 ERA as a reliever in 75 relief innings per year. What ERA would he need to produce in 200 innings as a starting pitcher to be equally as valuable to the Red Sox?

This requires us to account for two things: replacement level and leverage. The leverage issue is fairly straightforward. A typical closer has a leverage index of about 1.75, meaning that a run that he prevents has 1.75 times as much impact on the outcome of the game as a run prevented in the top of the first inning. (The average leverage index for the 23 pitchers with 20 or more saves thus far on the year is 1.74). A starting pitcher, by contrast, can be assumed to have leverage of exactly 1.

The replacement level issue is more complicated. One essential fact, as I reported in BP 2006, is that the typical pitcher will have an ERA about 25% higher when pitching in a starting role than when pitching in relief. That is, if you take a given reliever with a 3.00 ERA, your best guess, all else being equal, is that his ERA as a starter would be 3.75.

Does that mean that the average starting pitcher has an ERA 25% higher than the average relief pitcher? No, it does not. Over the past decade or so, ERAs of starting pitchers have run about only about 7% higher than relief pitcher ERAs.

Why the disconnect? The simple answer is that starters, as a group, are better pitchers than relievers. Starting pitchers, after all, are throwing the bulk of your innings, and so at the 30,000-foot level, it nearly has to be this way if baseball teams are making efficient decisions.

Thus, wherever we set replacement level for starting pitchers and relievers, I think we need to stick to the 5:4 ratio between the two. This is only fair, particularly if we are going to give relief pitchers credit for pitching high-leverage innings. It is simply not that difficult to come up with a pitcher who can turn in a decent ERA pitching one inning every second or third day. For purposes of simplicity, in fact, let's set replacement level at a 5.00 ERA for starting pitchers, and 4.00 for relievers. I recognize that this sets the replacement level bar a bit closer to league average than methods like VORP might assume, but considering my feelings on player valuation, this is somewhat intentional.

Throw all of these assumptions into a blender, and we find that a 2.00 ERA closer is roughly as valuable as a 3.69 ERA, 200-inning starting pitcher. For the sake of context, a 3.69 ERA would rank ninth among qualified American League starters, just behind Barry Zito:

              IP      ERA      Rep. Lev.      Leverage     Value
----------------------------------------------------------------
Reliever      75      2.00      4.00             1.75       29.1
Starter      200      3.69      5.00             1.00       29.1

The question, then, is whether Papelbon could post an ERA as good as 3.69 as a starting pitcher. Actually, that is not the only question. We also need to know whether Papelbon is capable of pitching 200 innings as a starter without substantially increasing his risk of injury. But let's stick with the rate performance question for now.

As I described before, the typical relief pitcher would wind up with an ERA about 25% higher if he converted to the starting rotation. Thus, we might assume that Papelbon's 2.00 ERA as a reliever would translate into a 2.50 ERA as a starter, which would certainly be the more valuable performance. But Papelbon is not a typical relief pitcher. Rather, he is in the 95th percentile for relief pitchers. There is simply not much precedent for a reliever of that quality being moved to the starting rotation. In fact, in only three cases in baseball history has a pitcher thrown at least 90% of his innings as a starter following a year in which he accumulated at least 25 saves and had an ERA of 3.00 or better. John Smoltz' conversion went well last season, but he had a history as an All-Star caliber starting pitcher. In the cases of Goose Gossage and Rick Aguilera, meanwhile, the results were fairly disastrous.

On the other hand, there are quite a number of cases in which starting pitchers--successful, unsuccessful, or somewhere in between--were converted to the bullpen. Just as the defensive spectrum is a one-way street, pitchers tend to move from starting to relief as their careers progress, and not the other way around. Next week, we'll take a detailed look at that group of pitchers to determine if there are any characteristics that make a pitcher especially well adaptable to a relief role. Hint: the answer is yes, and they have some pretty interesting implications for Mssr. Papelbon.

Nate Silver is an author of Baseball Prospectus. 
Click here to see Nate's other articles. You can contact Nate by clicking here