May 6, 2016

DRA 2016

DRA Returns!

by Jonathan Judge

The 2016 baseball season is underway, and now that we are a month into it, Deserved Run Average (“DRA”) is out of the gate as well.

Introduced last year, Deserved Run Average sets out to explain the runs a pitcher should have given up, rather than those that happened to cross the plate and be charged to him. (Because DRA relies on context, we have to wait a few weeks for that context to start establishing itself.) Other pitching run estimators tend to focus on the outcomes of plays, but DRA focuses on the likelihood that the pitcher was actually responsible for those outcomes. DRA does this by looking at the outcomes of each play in light of the particular players on the field, the type of event, and various factors that can influence those events, such as: stadium, catcher framing, temperature, and base-out state.

Much like the players it evaluates, DRA was determined to show up this year in the best shape of its life. And so we’ve made several changes under hood, some because they made the metric better, and others in response to feedback we’ve received.

Overview
Taking this from the top, the core features will be very familiar to you. To start, DRA remains the core measure of past pitcher performance, and the basis for pitcher Wins Above Replacement Player (PWARP) here at Baseball Prospectus. DRA is on the scale of Runs Allowed per 9 Innings (RA9).

Our scale metric of DRA– sets the average pitcher performance in each season to 100, and rates players by how much above or below average they are. DRA- and DRA are both fully context and park-adjusted, and will move in sync within a particular season. So, as of the time this article was written, the following are the top pitchers so far this year in DRA and DRA–:

Player	Team	DRA	DRA–
Dellin Betances	Yankees	1.63	74
Hector Neris	Phillies	1.95	79
Noah Syndergaard	Mets	2.10	81
Clayton Kershaw	Dodgers	2.10	81
Craig Kimbrel	Red Sox	2.24	82

Because it is scaled to RA9 each year, DRA reflects the current run environment. DRA–, on the other hand, is designed to be environment-neutral, and therefore functions as a way to compare pitchers from different seasons and even eras, to see how they stack up against each other. In other words, DRA– evaluates how well you performed against your peers, rather than how many runs were available to be given up.

So, since 1990, here are the top 10 pitching performances by DRA–: the accompanying DRA column reminds you why DRA– is the better metric for multi-season comparisons.

Season	Player	Team	DRA	DRA–
2000	Pedro Martinez	Red Sox	1.89	31
1999	Pedro Martinez	Red Sox	1.63	32
1995	Randy Johnson	Mariners	1.71	35
2004	Randy Johnson	Diamondbacks	1.75	38
2001	Randy Johnson	Diamondbacks	1.50	40

Now that you’ve seen what is familiar, let’s talk about what is different.

Here are the most significant changes:

Basestealing and Errant Pitches
Last year, DRA only considered the effect of basestealing and errant pitches[1] if those metrics affected the run-scoring environment as a whole for a particular season. This turned out to be both irritating and confusing to readers, who could not figure out if these events were being considered during a given season or not (in recent years: not).

We’ve decided that our original approach ended up answering the wrong question. It’s certainly interesting how basestealing ends up being more relevant in some run environments rather than others, but the point of DRA is to ask how particular pitchers are being affected at a given time by these events, not to make broad historical judgments about baseball run-scoring.

So, from here on out, basestealing and errant pitches will be reflected in all pitcher DRAs, all the time. This is actually easy to do, because we already publish runs gained or lost from our stolen base and errant pitch metrics. Because these events are modeled separately from the events that DRA models, we can just add or subtract the runs separately estimated from basestealing and errant pitches to the raw number of deserved runs derived from batting events before calculating the final DRA. From now on, that’s what we will be doing.

Avoiding Zero
Last year, early-season DRA values tended to veer into, and often remain in, negative territory. You can only hear so many jokes about Dellin Betances “breaking the system” before it becomes annoying. This problem affects all pitcher run estimators to some extent, but if you are properly modeling run-scoring it shouldn’t be happening at all.

We have now figured out a better way to model runs-allowed and have put that into practice with DRA. The method we derived has implications beyond DRA, and will be discussed in the second In Depth article to follow. For now, rest assured that DRA values will, at all times, truly adhere to a reasonable RA9-style scale, at the top and bottom of the range, with no more negative values — ever.

Pitcher Defense
A pitcher’s job is to generate outs, most of which will come from balls put into play inside the ballpark. And yet, too many people still believe that a pitcher has little, if any, effect on the likelihood that these events become outs. The most popular pitcher run estimator (FIP) expresses no opinion at all about these events, even though they are often critical to whether a team wins or loses a game.

Pitchers certainly do not have the kind of control over balls in play as they do over walks and strikeouts. But there are also pitchers who, particularly during a given season, display a consistent ability to control the likelihood of outs being generated on such plays, or limit the severity of the hits that do occur, even when we control for random BABIP luck and the quality of the fielders behind the pitcher.

Last year, pitcher defense was addressed indirectly, but this year it has a dedicated series of models. We specifically model the likelihood by which the pitcher is contributing to his own batted-ball results for putouts at various fielding positions, as well as for the seriousness of the hits given up, and give pitchers credit (or blame) for the modeled outcomes to the extent the models suggest is appropriate. These models do not currently incorporate Statcast fielder data, primarily because it is not publicly available. The models could easily accommodate the addition, although from the standpoint of pitcher evaluation, it’s not clear whether that would be a good idea.

Deserved Runs by Category
The most significant change is that we have moved from one model that considered all baseball batting events together, to a series of models that analyze each batting event separately. So, for example, home runs now have one model, strikeouts have another model, single-out putouts to each fielder have their own separate models, and double-plays beginning with infielders have their own models as well. There are 24 models in all at the moment, which is 22 more than last year.

This granularity is useful because factors like temperature and stadium are more important to home runs than they are to hit-batsmen, and catcher framing is more relevant to strikeouts than it is to routine flyballs. Last year, these events were all stacked together and modeled as one big group; going forward, they are being evaluated separately and their respective run values are then combined at the end.

Using this new approach, we determine the likelihood of a pitcher giving up each kind of event—whether it be a double, an infield single, a walk, or a groundout—and multiply that predicted probability by the linear weight of the event and the number of opportunities the pitcher had to generate that event.

By modeling the separate likelihood of each of these outcomes during each plate appearance, and then adding up their respective effects over the course of a season, we can more fairly give players credit for facing difficult situations, and similarly penalize them for facing easier lineups.

DRA Components, also by Category
Soon, we will also be rearranging the DRA Runs table to highlight these new components, which we have grouped for ease of interpretation into four general categories: (1) Hit Runs, (2) Not-in-Play Runs, (3) Out-Runs. As usual, negative is good (runs saved), positive is bad (extra runs allowed), and 0 is average.

Hit Runs
Hit Runs measure the pitcher’s ability to minimize damage on pitches that end up being hits of some kind. Pitchers with below-average ratings tend to give up extra-base hits and home runs when the bat hits the ball; pitchers with above-average ratings tend to yield weaker hits that cause less damage. Hit Runs is the sum of the pitcher’s run value in giving up home runs, triples, doubles, infield singles, and outfield singles, with each of those being separately modeled. This category tends to favor pitchers with heavy sinkers, deceptive deliveries, and other means of keeping the ball away from the outfield fences. As a counting stat, it also favors pitchers who throw a ton of innings and rack up outs. The best pitchers in Hit Runs in 2015 were:

Player	Team	Hit Runs Saved
Marco Estrada	Blue Jays	-11.0
Zack Greinke	Dodgers	-10.9
Jake Arrieta	Cubs	-9.1
Colby Lewis	Rangers	-8.1

The best Hit Run seasons since 1950 have been delivered by:

Season	Player	Team	Hit Runs Saved
1969	Larry Dierker	Colt 45s	-19.2
1996	Pat Hentgen	Blue Jays	-18.2
2006	Carlos Zambrano	Cubs	-17.3
2004	Randy Johnson	Diamondbacks	-16.6
1966	Sandy Koufax	Dodgers	-16.4

Not-In-Play (“NIP”) Runs
Not-In-Play or “NIP” Runs compile the results of models for unintentional walks, intentional walks, hit-batsmen, and strikeouts. These are the traditional power-pitcher categories. They are also by far the most valuable contributions a typical pitcher makes to run prevention. As a counting statistic, NIP Runs also reward innings volume along with quality. The best pitchers in Not-In-Play Runs for 2015 were:

Player	Team	NIP Runs Saved
Clayton Kershaw	Dodgers	-33.4
Max Scherzer	Nationals	-28.6
Chris Sale	White Sox	-27.6
Corey Kluber	Indians	-23.4

The best Not-In-Play Runs seasons since 1950 have been:

Season	Player	Team	NIP Runs Saved
1999	Pedro Martinez	Red Sox	-62.9
1999	Randy Johnson	Diamondbacks	-62.4
2000	Randy Johnson	Diamondbacks	-57.9
2000	Pedro Martinez	Red Sox	-55.4
2001	Randy Johnson	Diamondbacks	-55.3

You might say that 1999 through 2001 was an excellent time to make your point as a first-ballot Hall-of-Famer.

Out Runs
Out Runs refer to the pitcher’s ability to generate typical outs on balls in play, usually by generating weak or directional contact. These constitute the majority of baseball batting events.

We model the likelihood of a putout at each fielder position, controlling for the last recorded assist and other factors that seem to drive putouts at that position. We then fit the pitcher’s contribution to generating outs at each position. The skill set that helps generate outs is certainly similar to that which minimizes Hit Runs, although success in one does not guarantee good results in the other.

The best pitchers in Out Runs in 2015 were:

Player	Team	Single-Out Runs Saved
Colby Lewis	Rangers	-12.2
Marco Estrada	Blue Jays	-7.9
Chris Young	Royals	-7.8
Aaron Harang	Phillies	-7.6

The best Out-Runs seasons ever have been:

Season	Player	Team	Hit Runs Saved
1969	Larry Dierker	Colt 45s	-16.4
2002	Paul Byrd	Royals	-15.7
1954	Warren Spahn	Braves	-14.8
1993	Danny Darwin	Red Sox	-14.6
1988	Tom Browning	Reds	-13.7

The sum of these four categories ends up being the vast majority of the “deserved” runs allowed, so tracking them is a good way to see a pitcher’s strength and weaknesses. Within a few weeks, we will also offer a few of the categories people enjoyed last year, such as catcher framing, temperature, and role.

Contextual FIP (cFIP)
Contextual FIP (cFIP) is, as many of you know, our context-adjusted version of the Fielding Independent Pitching statistic. It tends to be more predictive than other pitcher run estimators, because it is both park and context-adjusted, and also incorporates shrinkage principles.

Last year, cFIP operated on its own set of models for home runs, walks, hit-batsmen, and strikeouts. The models were similar to the overall DRA model, but focused on individual components. This year, with DRA itself moving to individual component models, the two metrics now draw from some of the same models. cFIP will continue to focus only on so-called “true outcomes,” thus enhancing its predictive value, whereas DRA will remain focused on “all” outcomes to provide a comprehensive evaluation of player value.

* * * * *

For those who enjoy the deep dive into how DRA’s underlying architecture, and the statistical justifications for these changes, we welcome you to read the In-Depth article that accompanies this one.

As always we appreciate your feedback and suggestions.

Jonathan Judge is an author of Baseball Prospectus. Follow @bachlaw
Click here to see Jonathan's other articles. You can contact Jonathan by clicking here