In a postseason featuring late-inning heroics and a few coming-out parties for talented players (like Carlos Gonzalez), the actual game play has taken a back seat to shoddy umpiring. Instead of fans marveling at the dominant pitching performances from CC Sabathia and Cliff Lee, they have been left wondering how a play in which two runners were tagged while not standing on the base could only result in one out. Aside from the occasional missed call present throughout the regular season, the umpiring crews have blatantly blown calls in crucial situations that simply cannot be chalked up as “the human element” of the game and subsequently be brushed aside. Even worse, the crews have shown an inability to rectify mistakes made by individual members, and have refused to face the music and admit to their errors.
In an attempt to eradicate this issue and shift the focus back to whether or not the Phillies can defend last year’s title, Major League Baseball has decided that only veteran umpires will call the Fall Classic, a change from the past wherein younger umpires were rewarded with a ticket to the grandest of stages to garner more experience. The six chosen men in blue-Joe West, Gerry Davis, Brian Gorman, Mike Everitt, Jeff Nelson, and Dana DeMuth-will certainly have their work cut out for them, but how will the selection of these umpires affect the potential World Series hurlers?
Using a PITCHf/x database, I enlarged the strike zone to extend beyond the standard horizontal parameters as well as the specific heights for the hitters. From there, the number of called strikes in that inflated zone was measured in relation to the total number of called pitches-balls and called strikes. The higher the percentages in a certain game, the larger the strike zone. For all umpires with at least 10 games behind the plate, the league-average rate of called strikes out of called pitches with these parameters was 63.1 percent. When we bring in the six men calling the shots for the World Series, an interesting trend emerges, in that all but Jeff Nelson had a zone tighter than average for MLB umpires. With Nelson, the sextet averaged 62.2 percent; without him, 61.8 percent, suggesting this selected subset of umpires is much stricter in their definition of the zone. In terms of zone size, Nelson was a pitcher’s best friend, with West not far behind, while Davis and Everitt proved incredibly stingy, especially when stacked up to the league.
Since I measured this zone percentage for each umpire-game, standard deviations can be calculated as a shorthand method for determining consistency at holding true to a zone. As a whole, this group of 75 umpires averaged a standard deviation of 5.8 percent. Our subgrouped sextet boasted a 6.2 percent deviation, suggesting they were slightly less consistent than the league. Most of that falls squarely on the padded shoulders of Dana DeMuth who, at 7.1 percent, was the fourth least consistent of all umpires this season. The most consistent of the group, Mike Everitt, registered a 5.6 percent deviation essentially at the league average, so while the chosen ones were stricter than the league, their zones were also more prone to variation.
In addition to this data, what about more standard numbers aggregated with umpires behind the plate? The table below features the aforementioned inflated zone rate, the walk and strikeout rate of pitchers with these umps behind the dish, and the league ranks:
Umpire UBBr Rank ZONEr Rank SOr Rank
Joe West 9.12% 12 63.54% 27 18.50% 23
Brian Gorman 7.33% 66 62.36% 43 18.14% 30
Mike Everitt 8.28% 37 60.16% 64 18.11% 33
Dana DeMuth 8.94% 14 62.23% 46 17.61% 50
Jeff Nelson 9.30% 7 64.16% 14 16.72% 69
Gerry Davis 8.37% 35 59.64% 66 16.50% 72
MLB Average 8.27% 75 Tot 63.11% 75 Tot 17.95% 75 Total
One very important aspect to note is that the walk or strikeout rates for these umpires do not, on their own, tell us about the tendencies of umpires. For instance, look at Jeff Nelson, who has the largest zone of the group, but also the highest unintentional walk rate. And Gerry Davis, who maintains one of the tightest zones in the game, is essentially league average in unintentional walk rate observed. Running a correlation between a few of these metrics, zone size and walk rate produced an r-value of -0.53, which makes intuitive sense as larger zones should theoretically result in fewer walks being issued. Then again, Jeff Nelson serves as an example of how this will not always ring true. The level of consistency and size of the zone correlated at a rather insignificant 0.12, while consistency and walk rate share virtually no relationship at all.
Reverting back to our ultimate question, how does this information affect those potentially toeing the rubber this week? From the Phillies’ side of things, Cliff Lee will be on the mound in Games One, Four and, potentially, Seven. The Yankees will match Lee with Sabathia every step of the way, utilizing Burnett against Martinez in the second game while combating Hamels in the third game with fellow southpaw Andy Pettitte. The most interesting of the matchups-the one most likely to be affected by the umpires-is Burnett-Martinez, since the former is prone to wild spells and might not benefit from the men in blue if either Davis or Everitt is behind the plate. Similarly, Martinez now relies on acumen and location and the Phillies could be in for a long day if he isn’t getting calls. On the flipside, if Pedro receives the aid of a larger strike zone, expect him to exploit every quarter-inch of the zone in an attempt to carve up the Yankee hitters.
Then again, this subset of umpires was a bit less consistent at holding true to their zones than the league as a whole, so we may very well see DeMuth, the fourth least consistent ump in the league, become strict, while the much more consistent Nelson becomes impossible to gauge. The fact that the selection of these six umpires has become so publicized speaks volumes to the ineptitude that has been on display, and Major League Baseball better hope that their names become less prominent in the minds of fans after this Fall Classic or else face intense scrutiny throughout the entire offseason.
A version of this story originally appeared on ESPN Insider .
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
I suppose I was wrong, considering that so many comments have been left (although I suspect that has much to do with Eric's quick replies, which is certainly commendable) - my interest in baseball has to do with players, not umpiring.
Maybe it's just me, but this was very confusing. At first, it sounds like you are saying you did:
(called strikes within just the enlarged portion of the zone)
(all called pitches)
But that number clearly wouldn't be 63 percent. I guess what you did was:
(called strikes within the entire enlarged strikezone)
(all called pitches within the entire enlarged strikezone)
Is that correct?
That doesn't seem to necessarily indicated a larger strike zone...
You could probably do the inverse of this -- Called Balls within a smaller portion of the strike zone -- and then compare the 2 percentages. The larger the difference between the 2, the more inconsistent (and therefor lousier) the umpire.
Here's a few additional notes:
Within the standard strike zone, I get Nelson calling strikes on 77.54% of called pitches
Within the 2.5 inch-larger-in-all-directions strike zone, I get Nelson at a 64.31% rate.
So, a pitch within 2.5 inches of the strike zone has a 13.23% chance of being called a strike by Jeff Nelson. This is basically exactly in line with the league average (13.21%). This is because the rest of the league is better at calling pitches 2.5 inches or less outside of the strike zone a ball, but Nelson is above average at calling pitches in the zone a strike.
So perhaps Nelson's zone isn't much bigger than average?
Also, by pitch f/x pitch location, is it possible to determine the frequency that umpires miss calling their zone?
For the 2nd part of your comment, what exactly do you mean by calling their zone?
The second part is one that people, for whatever reason, don't seem to care about. I, personally, want to know WHY certain calls are missed, not if they are or not, because if we know the reason we can either alter the training of umps or re-train current ones.
If it's movement of certain pitches, where the catcher originally sets up, how the ump sets up, etc these are the questions to ask. Unfortunately we cannot answer them all yet.
However, the height of the strike zone can vary so maybe its best, for an initial test, to look at the width of an umpire's zone (since the width of the plate is fixed) and look at all pitches that are about three feet high (give or take three inches) which should be in the strike zone of any MLB player. Then, if that model holds, work more on defining the height of the zone.
Shouldn't umpires be studied with the same rigor as park factors?
This has happened before. A Canadian think-tank decided to use the publicly available standardised test scores (among other things) to evaluate schools in Ontario (both public and private schools). The government on Ontario really didn't like that, so they stopped making the data available.
MLB could do the same thing if they don't like how people use PitchF/X (though it blows my mind that anyone would think it wouldn't be used for this).
I hope some analyst produces regular umpire stats.
If the topic is uninteresting or too tedious, I can see why you wouldn't research umpires. But to be afraid of upsetting one group suggests potential restraint in research or bias.
I guess it'll just fall under the "not worth the effort/hassle" category?
All I really know about pitch F/X is from BP. I haven't been to any SABR conventions or pitch F/X conventions (though I heard there was a Hit F/X one a few months back). So, I was unaware about the context (i.e. the hesitancy, the powers that be were touchy) in which pitch F/X was provided to the public, and thus, apologize for my misunderstanding and for making inaccurate guesses at your intentions and your integrity.
Pitches inside standard strike zone: 76.1% Called Strikes
Pitches within 2.5 inches of the strike zone: 63.1%
Incorrect Calls / All Calls Made: 14.95%
When the Away Team is Pitching:
Pitches inside standard strike zone: 76.1% Called Strikes
Pitches within 2.5 inches of the strike zone: 62.7%
Incorrect Calls / All Calls Made = 14.48%
Inside the strike zone, umps are identical for home/away. Home teams get a small increase in called strikes for pitches near the strike zone.
Since every game is now televised and has an independent data collector monitoring it, my thought is that these people could mark the close plays and then someone could go back and look at the replays. On the ones that are too close to determine, give the ump the benefit of the doubt.
When looking at home plate umpires we must remember the following points:
1. Umpires are humans
2. Humans produce inconsistent work and make mistakes.
3. For the foreseeable future, computers will not call balls and strikes.
With this understanding, umpire's strike zone size and consistency should not generally (there are a few horrible cases) be judged outside of MLB. Instead, what should be looked at is how well pitchers and hitters adapt to each umpire's unique zone.
First the strike zone that umpires call isn't the perfect 2 by 2 box. The zone they call is much flatter and is shifted inside depending on the batter's handness.
I think umpires zones should be investigated, but not for evaluating the umpires. Instead, their strike zone tendencies should be known so it can be seen how well pitchers and hitters adapt to the different zones, which is actually a great part of the game baseball. Gamblers, who have more at stake than team pride, have been tracking umpire stats for years to see how they effect game scoring.
I have looked at a few cases of suspect umpire calls (Milton Bradley, Shane Victorino and Zack Greinke arguing about the zone) and in almost every case the umpires did nothing different than what they do every night. They call there own unique game and the hitters and pitchers that adapt first usually will have an advantage.