CSS Button No Image Css3Menu.com

Baseball Prospectus home
  
  
Click here to log in Click here to subscribe
<< Previous Article
Premium Article Collateral Damage: He'... (05/25)
<< Previous Column
The BP Wayback Machine... (05/19)
Next Column >>
The BP Wayback Machine... (06/02)
Next Article >>
Premium Article Prospectus Hit and Run... (05/26)

May 26, 2011

The BP Wayback Machine

How Do You Rate Relief?

by Nate Silver

While looking toward the future with our comprehensive slate of current content, we'd also like to recognize our rich past by drawing upon our extensive online archive of work dating back to 1997. In an effort to highlight the best of what's gone before, we'll be bringing you a weekly blast from BP's past, introducing or re-introducing you to some of the most informative and entertaining authors who have passed through our virtual halls. If you have fond recollections of a BP piece that you'd like to nominate for re-exposure to a wider audience, send us your suggestion.

Before Goose Gossage got into the Hall of Fame and Mariano Rivera reeled off another six superb seasons, Nate turned his statistical eye on the bullpen in the following article, which originally ran as a "Lies, Damned Lies" column on January 6, 2005.

Here at Baseball Prospectus and elsewhere, there has been a lot of bandwidth expended on just what it takes to make it to the Hall of Fame. The analysis has run the gamut from the sophisticated, WARP-based scores devised by Jay Jaffe, to the amusing but utterly arbitrary standards applied by Bill Simmons.

Truth be told, as much as I like Jay's work, I also think there is something to be said for gut-feel. A metric like JAWS tells you a lot about a guy's value, but it doesn't tell you quite as much about the shape of his career. JAWS applies what I would call the sausage method for assessing player value: you mush everything together into a nice, cylindrical package, add appropriate seasoning, and come out with what is hoped to be a tasty product. JAWS is, indeed, a very tasty sausage, and it's a heck of a lot more worthwhile than the spoiled cold cuts that most of the press is munching on. But it's still a sausage.

Personally, I think of Hall of Fame candidacies in terms of seasons. I look at Ryne Sandberg's record, for example, and see four elite seasons (1984 and 1990-92), two outstanding seasons (1985 and 1989), three good seasons (1986-88), four average seasons (1982-83, 1993, 1996) and a couple of none-of-the-aboves at the tail ends of his career. It helps to frame a guy's career to look at it in this manner; in Sandberg's case, I think it makes it clear that he's a Hall of Famer.

Applying this sort of season-based standard has the potential to be especially useful in evaluating the relief pitcher candidates on the ballot. For one thing, it has yet to really be established just what constitutes a Hall of Fame reliever and what doesn't; the standards applied so far have been inconsistent at best. For another thing, applying a sausage-based method to relief pitchers is problematic because of the question of leverage--we almost certainly ought to be giving some extra credit to seasons in which a substantial proportion of innings are pitched in situations in which the game is on the line, but just how much we need to adjust an open question. Defining what constitutes a "good" season, an "elite" season and so forth is also subject to a lot of arbitrary factors, but it at least creates a readily-understandable standard that can be applied to players of different positions and of different eras.

Now on to that arbitrary part. Here, are some fiddling, are the standards that I came up with in order to evaluate relief pitchers. I should rephrase that slightly: these standards are meant to evaluate closers. Both modern closers like Goose Gossage and post-modern closers like Billy Wagner should be handled adequately under this method. Pre-Holtzman relievers like Hoyt Wilhelm won't be, but those aren't the guys under Hall of Fame consideration at the moment.

Onward:

  • A "good" season is defined as a year in which:
    1. The pitcher has a park-adjusted ERA no greater than 75 percent of the league average
      -and he either-
    2. Accumulates at least 20 saves
      -or-
    3. Accumulates at least 10 saves AND pitches at least 100 innings.

     

  • An "outstanding" season is defined as a year in which:
    1. The pitcher has a park-adjusted ERA no greater than 65 percent of the league average
      -and he either-
    2. Accumulates at least 25 saves
      -or-
    3. c) Accumulates at least 15 saves AND pitches at least 100 innings.

     

  • An "elite" season is defined as a year in which:
    1. The pitcher has a park-adjusted ERA no greater than 50 percent better than the league average
      -and he either-
    2. Accumulates at least 30 saves
      -or-
    3. Accumulates at least 20 saves AND pitches at least 100 innings.

By this reckoning, there have been 511 "good" seasons, 270 "outstanding" seasons, and 112 "elite" closer seasons throughout baseball history. Are these standards perfect? Of course not, but in most cases, they should come reasonably close to what I'd call an "educated commonsensical" reckoning of the merits of a relief pitcher's season, which is what we're going for here. Danny Kolb, by our definition, had a good season in 2004; Eric Gagne had an excellent season, and Armando Benitez had an elite season. Note that these definitions are inclusive: if a player had an "excellent" season, he had a "good" season too.

Here are the leaders in each category since the dawn of time:

Good Seasons
---------------------
Lee Smith          10

Tom Henke           9

John Franco         8
Goose Gossage       8
Trevor Hoffman      8
Mariano Rivera      8

Rollie Fingers      7
Doug Jones          7
Bruce Sutter        7
Billy Wagner        7
John Wetteland      7

Rick Aguielera      6
Sparky Lyle         6
Hoyt Wilhelm        6

Armando Benitez     5
Dennis Eckersley    5
Gene Garber         5
Roberto Hernandez   5
Gary Lavelle        5
Mike Marshall       5
Jeff Montgomery     5
Randy Myers         5
Robb Nen            5
Gregg Olson         5
Troy Percival       5
Ron Perranoski      5
Dan Quisenberry     5
Dave Righetti       5
Dave Smith          5

 

Excellent Seasons
---------------------
Mariano Rivera      8

John Franco         7

Goose Gossage       6
Tom Henke           6
Doug Jones          6
Sparky Lyle         6

Roberto Hernandez   5
Lee Smith           5
John Wetteland      5

Dennis Eckersley    4
Keith Foulke        4
Trevor Hoffman      4
Robb Nen            4
Gregg Olson         4
Troy Percival       4
Dan Quisenberry     4
Bruce Sutter        4
Billy Wagner        4

 

Elite Seasons
---------------------
Mariano Rivera      6

Goose Gossage       5

Dennis Eckersley    3
Keith Foulke        3
Trevor Hoffman      3
Robb Nen            3
Bruce Sutter        3
John Wetteland      3

John Franco         2
Bryan Harvey        2
Tom Henke           2
Roberto Hernandez   2
Doug Jones          2
Lindy McDaniel      2
Jeff Montgomery     2
Randy Myers         2
Troy Percival       2
Ron Perranoski      2
Dan Quisenberry     2
Dick Radatz         2
Jeff Russell        2
Lee Smith           2
Billy Wagner        2

We can also, if we like, create a little sausage of our own by assigning one point for each good season, two points for each excellent season, and three points for each elite season. This should strike a pretty good balance between career and peak value.

Pitcher          Good   Excel   Elite   Points
----------------------------------------------
Mariano Rivera      8       8       6       42

Goose Gossage       8       6       5       35

John Franco         8       7       2       28
Tom Henke           9       6       2       27
Lee Smith          10       5       2       26
John Wetteland      7       5       3       26

Trevor Hoffman      8       5       3       25
Doug Jones          7       6       2       25
Bruce Sutter        7       4       3       24

Dennis Eckersley    5       4       3       22
Robb Nen            5       4       3       22
Keith Foulke        4       4       3       21
Roberto Hernandez   5       5       2       21
Sparky Lyle         6       6       1       21
Billy Wagner        7       4       2       21

Troy Percival       5       4       2       19
Dan Quisenberry     5       4       2       19
Randy Myers         5       3       2       17
Ron Perranoski      5       3       2       17
Rollie Fingers      7       3       1       16
Gregg Olson         5       4       1       16
Jeff Russell        4       3       2       16
Dick Radatz         3       3       2       15
Hoyt Wilhelm        6       3       1       15

Mariano Rivera is the rare example of a New York Yankee whose greatness might be underrated. He has a compelling case for being the best reliever in baseball history, and that is before giving him credit for his post-season success, his outstanding 1996 season as a middle reliever, or the balance of what he's likely to do in his career. He would be, in my mind, an easy first-ballot Hall of Famer if he were to retire tomorrow, and he sets the standard for closers both before and after him.

Gossage also stands out from the pack; that's a conclusion that a lot of analysts had reached through other methods, and it's validated here. Eight good seasons and five superb seasons ought to be enough to warrant a bronzing in Cooperstown regardless of a player's position.

Dennis Eckersley requires an asterisk; he didn't become a closer until he was 32 and his elite seasons were very, very elite. Sure, you'd like to have seen his numbers comport themselves a little bit more gracefully in the waning years of his career, but he's a Hall of Famer by any reasonable definition. Wilhelm requires an asterisk too, as we've already discussed. He doesn't simply predate the modern, one-inning closer; he predates the save rule entirely. More than 2200 innings with a career ERA+ of 146 makes him a Hall of Famer.

As for the rest of the relief pitchers--there really isn't a helluva lot of difference between the bunch of them. Lee Smith had more "good" seasons than any closer in history, but rarely reached elite status; he's no more a Hall of Famer than, say, Rusty Staub. Bruce Sutter doesn't stand out at all. His career is not appreciably different from Tom Henke's, and Henke was only named only 1.2 percent of ballots in 2001, his only year of eligibility. Giving Sutter extra credit for inventing the split-finger, a pitch that he did not actually invent, seems about as arbitrary as giving Henke extra credit for inventing funny-looking glasses. Rollie Fingers does even worse. It's likely that the system is underrating him slightly since his career stretches back to the late sixties, but Fingers has missed the mark by a lot. His inclusion in the Hall of Fame doesn't look particularly bad now, but I think it will go down as one of the lesser BBWAA selections of all-time in twenty years or so once we've had more time to digest what relief pitchers are capable of.

Trevor Hoffman's chances will go up radically if he can put together two or three more good seasons; last year was encouraging but I think the odds of that are unlikely. Billy Wagner is a relatively young 33, has been dominant enough in his best years, and will have a candidacy that ranks somewhere between credible and automatic if his arm holds up. It's too early to say anything about Eric Gagne, who has 12 points to his name thus far. I'm a big fan of John Smoltz in an Eckersley-lite sort of way, but he's accumulated just 7 points so far and would go in mostly for his work as a starter.

Henke, John Franco, John Wetteland and Doug Jones had better careers than they're generally given credit for. They aren't Hall of Famers.

The early returns suggest that it's awfully difficult for a relief pitcher to both pitch at an elite level for some period of time and to put together a long career. It might be because throwing absolutely as hard as you can on every pitch has a disproportionate and destructive impact on a pitcher's arm, or because making a lot of appearances is every bit as tough as pitching a lot of innings. It might be because relief pitchers are selected out for their lack of stamina. Or it might be bad luck. This will seem slightly obscure, but my sense is that, if you went back to the dawn of the closer era and put together an elaborate simulation considering player skill sets, aging patterns, and the statistical output that they produce, you'd wind up with five or six cinch Hall of Fame relief pitchers. As it is, we have three, counting Eckersley, and not counting Wilhelm, who predates the experiment.

While I think it would be short-sighted to banish relievers from the Hall because they don't throw as many innings as starting pitchers do, I also don't believe in positional quotas, and I certainly don't think that we should be doing relievers any special favors. With very rare exceptions, a Hall of Famer is someone who makes a significant contribution in many seasons, and an elite contribution in at least a few seasons. Goose Gossage meets that standard. Bruce Sutter and Lee Smith do not.

Nate Silver is an author of Baseball Prospectus. 
Click here to see Nate's other articles. You can contact Nate by clicking here

5 comments have been left for this article.

<< Previous Article
Premium Article Collateral Damage: He'... (05/25)
<< Previous Column
The BP Wayback Machine... (05/19)
Next Column >>
The BP Wayback Machine... (06/02)
Next Article >>
Premium Article Prospectus Hit and Run... (05/26)

RECENTLY AT BASEBALL PROSPECTUS
Playoff Prospectus: Come Undone
BP En Espanol: Previa de la NLCS: Cubs vs. D...
Playoff Prospectus: How Did This Team Get Ma...
Playoff Prospectus: Too Slow, Too Late
Premium Article Playoff Prospectus: PECOTA Odds and ALCS Gam...
Premium Article Playoff Prospectus: PECOTA Odds and NLCS Gam...
Playoff Prospectus: NLCS Preview: Cubs vs. D...

MORE FROM MAY 26, 2011
Fantasy Focus: Checking the Leaderboards
Premium Article Transaction Analysis: Return of the Prodigal...
Baseball ProGUESTus: Answers from a Sabermet...
Span and Sain and Pray for Rain: One Cliche ...
Premium Article The Payoff Pitch: How the Mets Were Lost
Fantasy Article Fantasy Beat: Value Picks in the Bullpen
Premium Article Prospectus Hit and Run: A Shuffled Deck

MORE BY NATE SILVER
2012-02-10 - The BP Wayback Machine: Neighborly Baseball
2011-12-15 - The BP Wayback Machine: PECOTA Takes on Pitc...
2011-08-25 - The BP Wayback Machine: Blowing It
2011-05-26 - The BP Wayback Machine: How Do You Rate Reli...
2011-04-05 - The BP Wayback Machine: Snowbound Schedule
2011-03-01 - The BP Wayback Machine: Wild Card: A Fairy T...
2009-05-07 - Premium Article Lies, Damned Lies: The Steroids Game
More...

MORE THE BP WAYBACK MACHINE
2011-06-16 - The BP Wayback Machine: Realignment
2011-06-09 - The BP Wayback Machine: Draft Pickin', Grinn...
2011-06-02 - The BP Wayback Machine: Baseball's Exciting ...
2011-05-26 - The BP Wayback Machine: How Do You Rate Reli...
2011-05-19 - The BP Wayback Machine: Baseball in 1864
2011-05-12 - The BP Wayback Machine: How to Read a Box Sc...
2011-05-04 - The BP Wayback Machine: The GM Starter Pack
More...