Baseball Prospectus Basics

Evaluating Defense

by James Click

Printer-friendly

It is one of the most suspenseful moments in a baseball game. There's a smash to the second baseman, he slides, knocks it down, picks up the ball, throws from his knees, and the first baseman can't dig it out. The crowds waits, and then the message appears on the scoreboard "On the last play, the official scorer has ruled: HIT."

Many of the problems inherent in evaluating defense are evident in the situation above. The first, and most crucial, is the fact that one of the most basic statistics involved in defense, the error, is assigned by one of baseball's loosest rules, left to the interpretation of the various official scorers. While the league has struggled for the past few seasons to remove the subjectivity inherent in calling the strike zone, it has done nothing to remove the same from the assignment of errors. Rules 10.05.a-e discuss in detail what is to be considered a "base hit"--essentially any ball that could not be fielded with "ordinary effort," a phrase that is never defined or clarified. In any field, statistics are only valuable if they are consistent and accurately reflect the action on the field. Errors, especially recently, have become assigned in such an ad hoc fashion as to relegate the statistic to nearly unusable status.

Second, the conclusions about defense performance we draw with our eyes are often based on insufficient or severely flawed evidence. Rey Ordonez made more than his fair share of exciting plays at shortstop over the past few seasons, but while they certainly look impressive, you must ask yourself if the same plays are made by other fielders more consistently without the all the spins and ninja rolls, either because of greater range or better positioning before the play. Distinguishing between the economically boring and the inefficiently flashy is simultaneously essential and difficult.

Finally, defense is intrinsically a team activity, thus metrics assigning individual applaud or fault can be as deceptive as RBIs and other team dependent hitting measurements. Statistics like Range Factor are largely based on the number of balls fielded by a particular player. However, the fact that a right fielder, for example, doesn't field as many balls as other right fielders in the league may be the result of many other factors besides actual defensive ability; he may be playing next to an extraordinary center fielder, in a park with a small or irregular rightfield area, or playing behind a pitching staff that gives up significantly fewer flyballs than a league average staff. As with virtually all metrics in baseball, the context is the key, not the actual statistic.

When analyzing defensive performance, the most important mental adjustment to make is to hold the defense accountable for every hit on balls that could have been fielded. Initially, this does not seem fair--defense and pitching are almost inseparably entangled. Determining if the causality lies with the pitcher or the fielders is even more perplexing than drawing conclusions about the play itself. This approach certainly requires some refinement, but it removes the decisions of the official scorer from the equation. Bill James suggested this approach in one of his Abstracts in the 1980s, calling the new metric "Defensive Efficiency (DE)." It is, quite simply, the percentage of balls in play fielded by the defense. The best teams are usually around .7300 with the worst around .6900.

While DE works fine for entire teams, it doesn't yield any information about individual defensive accomplishment. To that end, BP's Clay Davenport has developed several metrics that allow us to better evaluate each fielder. First and foremost, these metrics are based on range, complimenting players more for the plays they make than the plays they screw up. The reason for this is quite simple: there is no difference between reaching a ball and dropping it and letting it drop in for a hit. It's like the old axiom in golf: 97% of the putts that don't make it to the hole don't go in; in baseball, if you can't get to a ball, you can't field it. To determine the number of opportunities the player had to make a play, we start with the league average defensive performance at that position and adjust it for the player's particular situation. There are five key adjustments:

Park Factor: This affects outfielders more than other positions since infield size is strictly regulated while outfields seem to be turning more into amusement parks than playing surfaces. Other factors, such as the amount of foul territory, can also have profound effects on fielding performance, at least until baseball allows players to enter the stands to make plays. Imagine if the Cubs had built Wrigley Field with just a little more space in left field--Steve Bartman wouldn't have been able to reach out and grab his fateful ball.
Balls in Play: Certain teams allow more balls in play than others. Teams with staffs heavy with power pitchers (again, the Cubs come to mind) see many more strikeouts than an average team, yielding fewer balls in play. Fielders should not be penalized for not making an out when the pitcher does it all himself.
Ground balls and fly balls: Having the best outfield in the league means less if you've got a groundball staff. Since the GB/FB ratio listed for pitchers is based on outs, some slight adjustments have to made to estimate how many of the hits allows for ground balls with eyes and how many were liners and flies.
Left/right balance of the pitching staff: Left-handed pitchers face more right-handed batters--something that generally leads to more balls hit to short, third, and left field, and vice versa. After adjusting for strikeouts, better estimates of batted-ball distribution can be made.
Men of first base: With the exception of the rare triple play, the double play is one of the most impressive defensive feats in baseball. Obviously, though, certain teams have more opportunities for double plays than others, so knowing the raw number of double plays isn't nearly as helpful as knowing what percentage of double play opportunities were actually converted.

Using these, it's possible to estimate better the performances of individual fielders compared to the league average. From there, the number of runs allowed above or below average can be determined for each fielder. In Baseball Prospectus 2004, the "Defense" column for fielders contains values like "154-1B 3"; this indicates that the player played 154 games at first and was 3 runs above average (FRAA). You can also see these numbers on BP's PECOTA Cards.

Davenport Fielding Runs are a major step up from other widely available defensive statistics such as Fielding Percentage, Range Factor, and the enigmatic Zone Rating, but statistical analysis of baseball defense is still in its infancy. Front offices have access to much more advanced metrics than the public, some specifically charting where each batted ball is hit, how hard, and how high. As that wealth of data expands and reaches the public, it will be possible to better evaluate individual defensive performances. For the time being, however, just try to use an objective eye the next time ESPN's Web Gems comes on.