November 14, 2016

Prospectus Feature

The Cy Young and the Unfair Advantage of Defense

by Jonathan Judge

Printer-friendly

Contact Author

We’ve reached awards season, with the Cy Young—designated for the best pitcher in each league—due to be awarded this coming week.

In the National League, the named finalists are two Cubs (Kyle Hendricks, Jon Lester) and one National (Max Scherzer). Here is how they compare on various measures of pitcher quality:

With pitcher wins largely discredited, pitcher ERA has become the most common measurement cited by baseball writers in casting Cy Young votes. True to form, Hendricks had the best ERA among qualified NL pitchers this year, with Lester coming in second-best. The problem is that ERA is a flawed statistic, and this year’s Cy Young finalists make those flaws clearer than ever.

Many of ERA’s problems are not news. For one thing, ERA is biased toward groundball pitchers,[1] who put more pitches on the ground, thus inevitably causing more errors—which conveniently exempt those pitchers from responsibility for runs that cross the plate that inning. Perhaps for this reason, ERA is no better and maybe even less accurate than plain old unearned RA/9 when it comes to measuring pitcher ability.[2] Most concerning for our purposes, ERA is biased toward pitchers who have better defenses,[3] which results in those pitchers getting all the credit for outs primarily generated by defense, not pitching.

That’s a particular concern this year, since the Cubs have one of the best defenses in modern baseball history; since 1980, no team has held opposing batters to a lower batting average on balls in play (BABIP): .255. The bottom line is that while ERA may be useful to summarize what happened in an individual game, it is an inferior way to look at a pitcher’s season overall. To control for all these confounders, we need to turn to something else.

Last year, we created Deserved Run Average (DRA), which makes a number of adjustments designed to tease out a pitcher’s true contribution to his team’s success. DRA adjusts for parks, quality of opponent, catcher framing, and temperature, among other things. Most importantly for our purposes, DRA controls for the quality of the defense behind each pitcher. As of this year, DRA runs separate models for putouts at each position of interest, both for single outs (all positions) and for groundball double plays (positions two through six). That amounts to 14 models dedicated solely to evaluating pitchers in the context of their respective defenders, and that’s before we even get to things like home runs, walks, or hit batsmen.

The legitimacy of DRA as a comparative tool was tested and discussed earlier this year. From 2010 through 2015, DRA was more consistent from year-to-year, and better predicted the following year’s runs allowed than any other pitching metric. Particularly relevant for our analysis here, that same article showed that DRA’s outs models were four times more reliable than BABIP as a measure of pitcher skill for balls put in play. No one statistic should govern everything, but if ERA and DRA substantially disagree on a player,[4] chances are that it's the latter which is giving you the more accurate picture.

So, with DRA both explained and in hand, let’s return to our chart. Scherzer’s DRA and ERA are almost identical, indicating that his success is no mirage. The DRA-ERA gaps for Hendricks and Lester, however, are more striking. DRA says that Hendricks would be giving up more than an extra run per nine innings on an average defensive team. Lester’s gap is not so dramatic, but like Hendricks, his ERA might very well have a “3” in front of it rather than a “2” if he were pitching for some other team. Pitchers whose ERAs begin with 3 do not win Cy Young awards in this day and age, at least not when other qualified pitchers are posting better numbers, which they certainly were this year.

The last time a pitcher won a Cy Young with such a large differential between their ERA and their DRA was ... last year actually, when Jake Arrieta did so, featuring an ERA of 1.77 but a DRA of 2.89. Starting to see a pattern here? Arrieta rode a strong second-half narrative in 2015, getting credit along the way for being a wizard on balls in play--2.89 runs per 9 innings is still terrific pitching, but the Cubs' defense, particularly once Addison Russell was made the full-time shortstop, contributed substantially to the difference between the runs Arrieta “earned” in 2015 and those he actually “deserved.”

Before last year, at least in the NL, one has to go back almost 20 years—1998 to be exact—to find the last time a pitcher (Tom Glavine) had such a large gap between his ERA and DRA. This year Kyle Hendricks and Jon Lester both have a flashier pitching line than 1998 Tom Glavine, but the paths to their artificially low ERAs have a lot in common: put plenty of balls in play, and let an excellent defense do the rest.

One could certainly argue that defensive penalties like this are unfair, and that pitchers should not be penalized for pitching to the strengths of their team. A pitcher’s foremost job is to get outs, and if the best way to get outs is to get the ball on the ground, then that’s probably where a pitcher should be generating as many plays as possible. This may even have been the Cubs’ strategy this season, since they didn’t have a single starter reach even a strikeout per inning: an unusual fact for a championship rotation. Furthermore, tools like Statcast have certainly verified that generating weak contact is a skill, and Cubs pitchers have probably made at least some of their own luck in that respect.

There are two good responses to these arguments, at least in the context of the Cy Young award. The first is that there are plenty of pitchers who play in front of good defenses that nonetheless have good DRA numbers, with Tom Glavine’s own teammates being excellent examples. In front of the same good Braves defense, Greg Maddux (2.22 ERA, 2.08 DRA) and John Smoltz (2.90 ERA, 2.03 DRA) both contributed very good performances that did not benefit unduly from Braves defenders. DRA does give credit to pitchers who either substantially out-perform their team’s overall allowed BABIP (as Maddux did in 1998) or rely on other means to keep run-scoring down (as did Smoltz, whose .300 BABIP actually hurt him if anything).

The problem with Cubs pitchers is that their overall team BABIP was a ridiculously-low .255; working from this baseline, Hendricks came up with a slightly-improved BABIP of .250, and Lester basically made par at .256. The bottom line is that if a pitcher is uniquely responsible for his team’s success on balls in play, then it stands to reason (statistically and otherwise) that he should be substantially outperforming the rest of the staff in the BABIP department. When that is not happening, DRA properly credits the defense, not the pitching, for those results.

The second response is that the “E” in ERA means that only runs fairly charged to the pitcher are meant to be considered. And if we are going to credit a pitcher when his fielder screws up, it is only fair to debit that same pitcher when his fielders make him look even better than he really is. From a Statcast standpoint, while both Hendricks and Lester are above average at minimizing exit velocity, Jason Hammel, John Lackey, and most of the Cubs' bullpen were much worse—and yet the Cubs' defense still was turning an unusually-high number of balls in play into outs.

DRA was not computationally feasible 20 years ago, when Tom Glavine was being evaluated, but it is available now, and there is no reason to ignore the clear message it is sending about Cubs starters, and the fact that ERA just isn’t a fair measure of the Cubs' rotation. Measured by DRA, and certainly by strikeout rate, Jose Fernandez was the best starting pitcher in the National League. Since Fernandez unfortunately was not chosen as a finalist, the NL Cy Young probably should go to Scherzer. He deserves his 2.96 ERA, and unlike Hendricks and Lester, he accomplished it largely on his own: his .255 BABIP is more than 30 points lower than the Nationals' team average of .288.

Both Lester and Hendricks put an above-average number of balls on the ground this year, which creates a further ERA bias in their favor, and once again points to Scherzer as the more deserving candidate. Finally, Scherzer has many more innings pitched and many more strikeouts than either Hendricks or Lester. By any reasonably informed measure, Scherzer was the best pitcher of the three NL finalists.

Kyle Hendricks and Jon Lester are terrific pitchers who played key roles in ending the Cubs' 108-year championship drought. But the Cy Young is an individual award, and neither player was the best pitcher in the National League this year.

Jonathan Judge is an author of Baseball Prospectus. Follow @bachlaw
Click here to see Jonathan's other articles. You can contact Jonathan by clicking here