June 13, 2012

Overthinking It

Who Is the Best Umpire?

by Ben Lindbergh

Printer-friendly

Contact Author

While many of the most memorable umpire mistakes have come on force plays, tag plays, and “boundary calls,” the most common kind of blown call, by far, happens behind home plate several times a game. It’s possible to watch a game and forget about the base umpires, as long as none of them makes a glaring error. But it’s impossible to ignore the home plate umpire, who has to making a ruling on every single unstruck pitch. That’s why arguing balls and strikes leads to an automatic ejection—there are simply too many of them to make arguing each one permissible. Moreover, the strike zone is such a core component of baseball that questioning its consistency calls the integrity of the game into question.

Grousing about umpires is as old as the game itself, but the advent of instant replay—and more recently, ball-tracking technology—has made those complaints more numerous and provided conclusive evidence of occasional umpire incompetence. That doesn’t necessarily mean that we’re ready to do away with umpires, even if Major League Baseball would allow it. Even Mike Fast, a former Baseball Prospectus and current Houston Astros analyst who made his name by studying the data collected by Sportvision’s PITCHf/x system, has acknowledged that some significant technical hurdles would have to be cleared before an automated system could make more accurate calls in real time than human umpires. However, that hasn’t stopped, or even slowed, the steady stream of complaints about officiating coming from couches and clubhouse alike.

Red Sox manager Bobby Valentine is the latest to make his feelings felt. Last weekend, the Nationals swept the Red Sox at Fenway Park, as first Stephen Strasburg, then Gio Gonzalez, then Jordan Zimmermann, silenced Boston’s bats, holding the team to a total of nine runs in the three-game set. The Nationals’ starters have a collective 2.94 ERA: holding opposing teams to two or three runs is something they’ve been doing all season. Nonetheless, Valentine believed that in this instance, they had some help behind home plate, saying, “I thought [the Nats pitchers] pitched well. I thought they got pitches. In key situations. That weren’t strikes.”

Valentine went on to describe what he wants to see:

The game is simple. Throw it over the plate, call it a strike, don’t throw it over the plate, call it a ball. Simple. That’s all. It’s all anybody asks.

Of course, it’s not actually simple—if it were, no umpire would rule incorrectly, and calls for robot umps would be few and far between. Calling balls and strikes is extremely difficult. Even Valentine is aware of this. A day after his initial complaints, he summarized the difficulties umpires face:

I think they're very well trained, and I think they're very good at what they do. I think it's almost impossible to do what they do, so why do we ask them to do the impossible? If in fact you can't see the ball the last five feet, and now pitchers are throwing pitches that are moving in that zone, cutting and splitting and moving in the zone, your eye can't see what's happening. They're humans. We're asking humans to do a feat a human can't do.

But Valentine’s acknowledgement of the difficulty of calling pitches doesn’t make him any less eager to see them called correctly.

I want a ball called a ball and a strike called a strike, and figure out how to do it… That’s what the game is… Our game is not somebody else’s strike zone. Our game is what the book says. That’s how it should be played from Little League to Cooperstown, to make it fair, to make it right.

Much as Valentine might want it, a significant change in the way balls and strikes are called seems still to be a ways away. So who are the best of the old-fashioned, flesh-and-blood umpires when it comes to calling pitches correctly?

In the PITCHf/x era (2007-2012), 81 umpires have been behind the plate for at least 5000 pitches recorded by Sportvision’s system. Among these experienced umps, 86.8 percent of pitches have been called correctly, using the rulebook definition of the strike zone’s horizontal boundaries and a method for defining the top and bottom of the zone developed by Fast. The range between the best and worst umps seems fairly small: fewer than five percentage points separates the best from the worst. However, given the quantity of pitches a home plate umpire calls—since 2010, an average of roughly 81 per team per game—a difference in accuracy of even a few percentage points can make a major impact over the course of a single contest, let alone a whole season. In a game featuring an average number of called pitches, the difference between the best and worst umpires would be something on the order of seven correct calls.

The following 10 umpires to rule on at least 5000 pitches—all of whom are still active—have called the highest percentage of pitches correctly:

McClelland, the most accurate of the experience umpires, is also one of the most experienced—he was the ump who called George Brett out in the so-called “Pine Tar Game Game” in 1983, McClelland’s first full season. According to his MLB.com bio, the crew chief has worked at least one game at the major-league level in 31 seasons, the third-highest total among active umpires. At 6’6”, McClelland has different perspective on the zone than most umpires. He’s also known for his deliberate pace behind the plate, which makes the broadcaster’s job more difficult but might lead to more accurate calls. McClelland’s consistency hasn’t gone unrecognized: he has consistently finished at or near the top in player polls about baseball’s best umpires. Last month, the 60-year-old told the ​Des Moines Register ​that he was contemplating retirement. McClelland has made his share of controversial calls, but the game would be worse off without him.

Ironically, the three umpires who were behind the plate for the Nationals-Red Sox series—Alan Porter, Dana DeMuth, and Fieldin Culbreth—have all been above-average at calling balls and strikes. The trio probably missed some important pitches over the weekend, but in those three games, 87.1 percent of pitches to Red Sox batters were called correctly—an above-average success rate. That would likely come as little consolation to Valentine, whose team still couldn’t score.

All umpire data provided by Dan Brooks. Tabular data and heat maps for each umpire are available at Brooks Baseball.

A version of this story originally appeared on ESPN Insider .

Ben Lindbergh is an author of Baseball Prospectus. Follow @benlindbergh
Click here to see Ben's other articles. You can contact Ben by clicking here