November 25, 2015

Prospectus Feature

Updates to FRAA, BP's Fielding Metric

by Harry Pavlidis, Rob McQuown and Jonathan Judge

Our objective at BP Stats is to provide sophisticated and useful metrics. Our mission is to do this while working transparently. Part of this involves reviewing what we have. As much—if not more—is about building new things.

Today we arrive at an update to FRAA—Fielding Runs Above Average—that represents both lanes of the process. FRAA is the metric we use at BP to measure the defensive contributions made by players in the field. In reviewing what we had, we found something we really needed to fix—outfield assists were not being counted in FRAA. We’re also adding some new metrics for catcher throwing skills (SRAA and TRAA) along with the framing and blocking metrics (CSAA and EPAA) we already were adding. This will allow FRAA to reflect player defense across a variety of important spectrums.

All of these improvements are happening even while we work on the next generation of FRAA using some of the techniques that have produced things like DRA and CSAA. This new version of FRAA will be the next “old” version if our offseason projects go well.

Zoom Out: WARP

A quick review of how we make our Wins Above Replacement Player (WARP) metric. Here’s how we generate WARP, in two parts.

BWARP = (BRAA+BRR+BRR_ARM+FRAA+POS_ADJ+REP_LEVEL)/RPW

BRAA = Batting Runs Above Average

BRR = Baserunning Runs

BRR_ARM = Fielder arm ratings

FRAA = Fielding Runs Above Average

POS_ADJ = Positional adjustment

REP_LEVEL = Replacement level

RPW = Runs per win

PWARP = DRA converted into WARP scale.

WARP is BWARP+PWARP.

Notice something? Fielder arm ratings are not part of FRAA, only WARP.

Arms and Fielding
BRR_ARM is supposed to take into account "fielder arm ratings"—everything from outfielder throws to catcher SB/CS-against to infielders allowing base advancements. But a) it has some quirks that make it such that it's not adequately representing what happened historically, and b) it's not currently included in FRAA, which makes FRAA an incomplete metric for defense.

The basic concept of BRR_ARM is to take the RE_DIFF before and after for plays where a fielder is involved. In this, it's probably best measuring infielder arms, so we're first replacing catcher throwing and outfielder assists. This will result in:

* Infielders will see their FRAA change, based on how good their arms are. Their WARP will be unaffected.

* Outfielders will see their FRAA change, and their WARP change to a lesser extent (BRR_ARM was only capturing a small portion of the runs saved by outfield assists, so high-assist players will see big changes).

* Catchers will see their FRAA change, and their WARP change by a different amount.

Outfielder assists are evaluated using a new method. Each event has a different run impact depending on the base-out state (RE24). This value is compared to the actual change in run expectancy (RE, and including actual runs scored) that takes place on each “assist opportunity” play to determine the number of runs which were either a) prevented by that assist (often a sizeable run impact), or b) allowed by the failure to record an assist (a very small run value). These runs disrupted a league-season neutral FRAA adjustment for outfield assists.

Catchers' throwing contributions are now being measured by SRAA (Swipe Rate Above Average) and TRAA (Takeoff Rate Above Average). The former measures base-stealing success, the latter measures base-stealing attempts, both of which affect the game. You may be familiar with these measures for baserunners and pitchers, but now we’re including SRAA and TRAA as add-ons to FRAA for catchers. TRAA is a small component, providing less than a run plus-or-minus, while SRAA stretches out toward five or more runs plus-or-minus. No surprise there, as throwing runners out is obviously more important (and detectable) than intimidating them back to the bag.

SRAA is, like FRAA, a work in progress. As you peruse the catcher data, you may notice 2011 is strangely close to zero for all catchers. This is something we have tentatively addressed using a provisional model which doesn’t “degenerate” for 2011. (For the uninitiated, an explanation of degenerate models.) It also provides conservative estimates and is too computationally intensive to be a production process. As we work on improving the model and tuning its performance, we plan on updating the current method.

Blocking and Framing Updates

We’ve recently discussed our revisions to Errant Pitches Above Average, or EPAA, our second biggest contributor to catcher FRAA in our new method.

Framing provides the biggest potential impact for a catcher’s defensive contribution. When merging the CSAA data into FRAA we found an overcount of “extra strikes.” However, after harmonizing the two (we had 99.5 percent accounted for), we found little or no change at the top end of the rankings. The bottom end dropped back down...apparently this is where our extra strikes were going. Once viewed as a possible feature (conservative estimates of negative framing impacts) it was truly a bug and is now corrected. Apologies to Ryan Doumit.

Zoom Out: WARP Revised

The end result is a top-line BWARP ‘formula’ that looks mostly the same, but actually includes a handful of new and improved items.

BWARP = (BRAA+BRR+FRAA*+POS_ADJ+REP_LEVEL)/RPW

*includes OF assists, catcher CSAA, EPAA, SRAA and TRAA

PECOTA
This will also show up in our projections, which have been missing OF assists—potentially skewing our assessment of a team's defense. The new additions, particularly around catcher defense, are bound to impact our looks ahead.

The first batch of PECOTA projections for these new metrics can be found in the 2016 Annual. So go preorder one.

See it Now
You can review the updated FRAA estimates for individual player seasons here. The components for catchers are available here and here.

Harry Pavlidis is an author of Baseball Prospectus. Follow @harrypav
Click here to see Harry's other articles. You can contact Harry by clicking here
Rob McQuown is an author of Baseball Prospectus. Follow @robmcquown
Click here to see Rob's other articles. You can contact Rob by clicking here
Jonathan Judge is an author of Baseball Prospectus. Follow @bachlaw
Click here to see Jonathan's other articles. You can contact Jonathan by clicking here