February 25, 2008

Schrodinger's Bat

What Would Bacon Do?

by Dan Fox

the archives are now free. All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible. Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month Recurring subscription - cancel anytime. Subscribe for $39.95 for one year a 33% savings over the monthly price! Purchase a $39.95 gift subscription a 33% savings over the monthly price! Already a subscriber? Click here and use the blue login bar to log in.

"It is the peculiar and perpetual error of the human understanding to be more moved and excited by affirmatives than by negatives."
--Francis Bacon (1561-1626)

Since everyone else seems to be chiming in on the fielding prowess of Derek Jeter, perhaps it's relevant to pause and ask the question, what would Sir Francis Bacon, the English philosopher and statesman, have thought of Jeter's defense? That question may be more relevant than it would first appear since not only did Bacon popularize what we now call the scientific or Baconian method, but he also, in his wildly ambitious quest to categorize all of human knowledge in a 1620 book called Novum Organum (meaning 'New Instrument'). In it, he classified intellectual fallacies into what has been called "the four idols." Both his method and Novum Organum have a bearing on the question of Jeter's ability at shortstop.

First, of course, there is the question of whether one can objectively evaluate Jeter's defensive ability, or that of any shortstop. The question of defensive analysis has been a stickler for most of major league history due primarily to the lack of creativity in determining what to track; the same six official fielding stats—games played, total chances, putouts, assists, errors, and fielding percentage—adopted by the National League in 1876 are the same six in use more than 100 years later. However, the ease with which new kinds of data can be collected and analyzed in recent years has made the task far easier.

Building off of a wealth of new data that includes play-by-play records of each and every of the almost 200,000 events in a major league season complete with observational recording of each batted ball and its location on the field, researchers have constructed sophisticated systems (although based on straightforward and accessible ideas) that probabilistically debit and credit fielders for making or not making plays judged as similarly difficult, as compared to their peers.

University of Pennsylvania professor Shane Jensen's Spatial Aggregate Fielding Evaluation system (SAFE) has certainly grabbed the attention of the media after he revealed at a meeting of the American Association for the Advancement of Science in Boston that Jeter ranks at the bottom of the list of shortstops in the 2002-2005 period, averaging nearly -14 runs per season. Only Michael Young of the Rangers was even close, at -13 runs. But Jensen's system is only one of several built on similar principles, all of which rank Jeter poorly, and some of which are shown in the following table. A quick confession: while also relying on play-by-play data, SFR does not use as fine-grained data as the other systems, but I place it hear alongside these other metrics to make a comparison.

 

Year  SAFE    UZR     PMR      Plus/Minus     SFR
     (Runs)  (Runs)  (Outs) (Balls fielded)  (Runs)
2003   -16    -21     -13         -14          -15
2004   -11      3     -30         -16            2
2005   -19    -11     -41         -34           -9
2006   n/a    -15     -18         -22           -8
2007   n/a     -7*    -40         -34          -20
* First half of the season only

There is clearly a decent amount of variation in the numbers from year to year and from system to system (even taking into account the fact that PMR and Plus/Minus are based on balls fielded, while the others are calculated in terms of runs). But even so, a clear trend emerges from the table—Derek Jeter simply does not rate well in these systems when compared to his peers.

Bacon was one of the first observers to realize what the trend in the previous table might signify. After all, in Novum Organum he also had this say about his new-fangled inductive method of reasoning from observation to fact, rather than from general principles to particular truths:

There are and can be only two ways of searching into and discovering truth. The one flies from the senses and particulars to the most general axioms, and from these principles, the truth of which it takes for settled and immovable, proceeds to judgment and middle axioms. And this way is now in fashion. The other derives axioms from the senses and particulars, rising by a gradual and unbroken ascent, so that it arrives at the most general axioms last of all. This is the true way, but as yet untried—Aphorism 19

In the case of Jeter, those "particulars" of the various fielding systems do indeed tend to rise in a "gradual and unbroken ascent" to arrive at the general axiom that the Captain is simply not the defensive wizard that his Gold Gloves might otherwise indicate he is.

Of course Bacon might go further, and wonder whether, in designing the "particulars" of the various systems, these researchers have made accurate-enough observations to warrant the inductive conclusion. Both Jeter and his defenders have picked up on this idea several times, with Jeter most recently noting:

Every [shortstop] doesn't stay in the same spot, everyone doesn't have the same pitching. Everyone doesn't have the same hitters running, it's impossible to do that.

And with Yankees senior advisor Gene Michael observing:

First of all, what pitching staff was out there? Each team has a different staff. Derek doesn't really have a sinkerball pitching staff, whereas other shortstops, you sit behind certain pitchers, you're going to get a lot of ground balls... You simply can't do that by those charts, that's a bunch of baloney... You have to use a scout's eye to determine range.

There is certainly something to consider in these criticisms. For example, none of the systems take into account the starting position of the fielder on individual plays, and so are making an assumption that over the large number of observations—a shortstop would be expected to field somewhere between 350 and 425 balls a year—positioning evens out. That said, the systems also assume (not unfairly in my view) that positioning is a part of what being a good defender is all about. So, if a fielder is claiming that his positioning is making him look bad defensively, which boils down to his not making as many outs in the field as he should, thereby costing his team runs and wins, it's incumbent upon him to change whatever failed strategy is being employed.

These systems do variously take into account a large amount of context upon which these criticisms are based, however. From the distance the ball traveled, to both how the ball was hit (grounder, line drive, fly ball, popup) and how hard, to the number of outs and the occupied bases (which affects positioning), to the handedness of the batter, to the ground-ball tendency of the pitcher on the mound, and even to the park itself, context is carefully accounted for. No disrespect to Gene Michael intended, it appears you actually can "do that by those charts." In fact, in what may come as some consolation to Jeter fans, breaking down the data in this fine-grained fashion reveals that several of the systems see Jeter as an above-average fielder on balls in the air.

Even so, critics might further contend that some of that context is recorded by human observers, so it is therefore subject to bias and simple human error, anticipating one of Bacon's favorite themes (which will be further developed in a bit). That too is a fair criticism, although it should be remembered that for the various metrics shown in the table above, there are multiple forms of data collection represented, and that each employs multiple observers (typically based on park), and that there is turnover in those positions. As a result it seems highly unlikely that over the course of several years and thousands of balls in play there exists some kind of systematic bias that works against Jeter.

Perhaps the final word on the objective analysis of Derek Jeter's defense was had by Tom Tango. In a recent essay from the most recent The Hardball Times Annual, Tango uses a more direct method of neutralizing those factors discussed above by comparing the percentage of outs on all balls in play with and without Jeter on the field in a variety of scenarios. For example, with Jeter on the field the shortstop makes an out on 11.6 percent of balls in play. However, when looking at all pitchers that Jeter has played behind when pitching with other shortstops on the field, the rate goes up to 12.5 percent—that's a difference of 38 plays over a full season, and the second-worst mark for a regular shortstop in baseball, behind only Young. Tango then does likewise controlling for batters (Jeter is 25 plays worse, fourth from the bottom), a runner on first base (11 plays worse, ahead of only Felipe Lopez), and park (18 plays worse, ranking in the bottom half).

Indeed, the evidence is such that Bacon could only conclude that Jeter is not even an average defender. What then accounts for the vast difference between many of the subjective views of Jeter's abilities, and the objective view, as represented by the advanced fielding systems?

Here's where Bacon again may offer us a clue. In Novum Organum he identified several impediments, which he called idols (aphorisms 38-44) to human reasoning in the pursuit of a correct interpretation of nature. One of the idols he defined, "Idols of the Tribe" (idola tribus), are impediments grounded in ways of thinking common to the human species. Of these Bacon says they are:

inherent in human nature and the very tribe or race of man; for man's sense is falsely asserted to be the standard of things; on the contrary, all the perceptions both of the senses and the mind bear reference to man and not to the universe...

One might say that idols of the tribe reflect weaknesses in reasoning that are common to all people. Among those weaknesses can be included the problem of affirmation bias, looking for only those examples that reinforce our preconceived opinion, and the related inability to deal with large sample sizes only through observation.

Related to this first weakness, in support of their man fans of Jeter might point to the way which he makes hustling, diving plays (who can forget his dive into the stands against the Red Sox on July 1, 2004, or "the flip" in the 2001 ALDS?) and his patented jump-throw from deep in the hole at short. Leaving aside the actual efficiency of the latter (about which Bill James touched upon in a fine essay in The Fielding Bible, and David Pinto discussed more recently), the common thread is that both are infrequent and isolated instances of Jeter's fielding ability. Bacon recognized that such positive events, although rare and not necessarily indicative of a general trend, can strongly color our overall perception. While this need not always be the case, in the case of Jeter one suspects this plays a role.

Looking only for affirming examples is actually a product of the second weakness, where we are unable to discern real differences in results when those differences are spread over so many individual observations. For example, who can tell a .300 hitter from a .270 hitter by observing even several hundred at-bats from the two? Should we be less surprised that most of us can't tell the difference between a good and bad shortstop, even after faithfully watching them all season? Our minds are simply not designed to collect, categorize, and compare at that level of detail. Not to put too fine a point on it, in contrast computers are ideally suited to do so.

Ultimately, we all have to make our own decisions about how we want to look at the world. For my money and understanding, and in full appreciation of the frailties of human reasoning and the power of inductive reasoning, I'd have to conclude that while Francis Bacon would appreciate the many wonderful qualities of a superb player like Derek Jeter—especially his leadership, dedication, baserunning, and offensive prowess, to name but a few—Bacon would have to conclude that when compared with other major league quality shortstops, when it comes to defense, Jeter leaves something to be desired.