August 11, 2004

Lies, Damned Lies

Deconstructing General Managers

by Nate Silver

Printer-friendly

Contact Author

I was watching the Red Sox game on Monday night when Orlando Cabrera came to the plate. Cabrera was the starting shortstop. I did a quick double-take, then browsed through half a UTK column for news of Nomar Garciaparra's latest injury before realizing...

The Garciaparra deal is old news now, and both Joe Sheehan and Chris Kahrl have done their usual tip-top job of breaking the trade down. But perhaps because the Cubs were the lucky beneficiaries of Theo Epstein's misstep, or perhaps because it's just so fricking strange to see Garciaparra in another uniform, like something from an EA Sports bizarro world, I've found myself thinking about the trade a lot since it was consummated.

It's fair to say that I agree with Joe's conclusion: the trade was made for non-baseball reasons. Contrary to the obvious comparison, this deal was not Jeremy Giambi/John Mabry, Part Deux, in which a marginal player was replaced by a sub-marginal one. Garciaparra, though no longer a star, is still a tremendously valuable commodity, probably worth five or six wins above a replacement-level shortstop over the course of a full season with his bat. Losing him will have a material negative impact on the Red Sox' chances of reaching the post-season.

The more compelling question is not, "Why make the trade?", but rather, "Why would Theo Epstein make a trade for non-baseball reasons?" I don't mean that facetiously. The deal with the Cubs reveals, in some shape and form, certain of the limitations inherent in the analytical movement and its applicability in baseball front offices:

1. GMs are people, too. General managers are not immune from making decisions based on "soft" factors like interpersonal considerations. On the contrary, as anyone who has ever met one can attest, baseball executives are selected out in substantial part for their people skills. The better GMs--a group in which Epstein belongs, as does Jim Hendry--thrive because they combine those people skills with strong analytical abilities.

   It is unrealistic to expect to find a general manager who is able to completely divorce himself from the sort of "irrational," right-brained, human-based factors that influence decisions in all walks of life. Somebody with that personality type would probably never have been deemed fit for the GM role in the first place. It is natural to expect a GM to be sensitive to the opinions of people within his organization, contemporaries outside of his organization, fans and media members. (The difficult experience of Dan Duquette, who was lambasted in Boston for appearing to be such a person, is instructive here.)

2. Major-league decision makers have incentive problems. The example I like to provide here does not involve a general manager, nor does it involve baseball at all. Bear with me.

   Two seasons ago, in the midst of a predictably dismal season for the Detroit Lions, coach Marty Mornhinweg, elected to kick off rather than receive the ball when his team won the coin toss in overtime in a late November game against the Bears. The decision was a perfectly logical one. Since the start of overtime play in the NFL, the receiving team has gone on to win the game something like 52% of the time. The intrinsic advantage of receiving the ball is very small. In this particular game, there was a stiff wind, which the Lions could have at their back for the entire overtime period. Moreover, the Bears' defense that season was much better than their offense, and the Lions would very probably have received favorable field position if the Bears had been stopped and forced to punt. A simulation would have found that Mornhinweg's decision was clearly correct from the standpoint of giving the Lions the best chance to win the game.

   But it was also highly unconventional: only a handful of coaches in NFL history have won the coin toss and elected to kick. As it happened, the Bears scored on the first possession. Mornhinweg was fired after the season. Had Mornhinweg done the usual thing, and had his team receive, and the Lions went four-and-out, and the Bears won on a wind-assisted field goal shortly thereafter, nobody much would have noticed: it would have been just another loss in the midst of a 1-15 season. Mornhinweg's decision was stupid, not from a football standpoint, but from the standpoint of his job security.

   Baseball general managers, with the apparent exception of Chuck LaMar, are not Supreme Court justices. They have an incentive--call it the Jim Duquette Problem--to trade off long-term success for near-term success, even if is not in the ultimate best interest of their organization. They have an incentive--call it the Marty Mornhinweg Problem--to follow the conventional line of decision making, which means maintaining the status quo when the team is performing above expectations and altering it when the team is performing below them. Finally, they have an incentive--call it the Larry Bowa Problem--to avoid selecting courses of action that could call into question the caliber of their previous decisions.

3. "Blind spots" in statistical analysis provide room for misinterpretation. One of the things that Baseball Prospectus has gotten better at over the years is in acknowledging those areas in which performance analysis does not provide all the answers. Something like the defensive contributions of individual players is an obvious example; we can quantify a player's performance, but only up to a certain degree of resolution. If we want to dispute, say, a Gold Glove selection, we certainly have some ammunition to work with, but nothing with the deadly AK-47 precision of VORP or SNWL. Pitch counts probably belong in this category. Player forecasting does, too; PECOTA explicitly provides for a confidence interval around each player's forecast.

   The problem, of course, is that it is easy for people to replace the "explanatory vacuum" that this ambiguity creates with their own subjective preferences. On a couple of occasions, for example, I have had discussions with major-league personnel who use the presence of a forecast range within PECOTA as surefire evidence that their overoptimistic perception concerning one of their own players is correct.

   We encounter a similar problem with pitch counts.

   So yer sayin' that 20 percent of pitchers who're worked hard git hurt, and 10 percent of pitchers who ain't worked hard git hurt? Our boy's built like an ox. He ain't in that 20 percent. Just look at 'im out there. Built like an ox, I says.

   I don't mean to suggest that you ought to never consider factors that are outside the scope of something that a statistical index provides. It's just that one needs to be careful. Note the difference, for example, between the following two statements:

   • "We think that this LaLoosh kid is going to do much better than his PECOTA forecast because he has a good cut fastball."

   • "We think that this LaLoosh kid is going to do much better than his PECOTA forecast because he developed a good cut fastball in Venezuela this winter."

   In the former case, any benefits arising from the pitcher's good cut fastball ought to already have worked their way into his statistical record, and therefore into his PECOTA forecast. In the latter case, we have new information that PECOTA hasn't accounted for. We can debate the credibility and the importance of this information, but it is at least plausible that it could have a material and unaccounted for impact on a player's valuation.

   This might seem like a clear enough distinction, but it is easy for a decision-maker to use the ambiguity intrinsic in baseball analysis as a license to insert subjective explanations that are otherwise consistent with his organizational philosophy, or with his own incentives. This tendency probably occurs at least partly on a subconscious level. Finally, it is possible that this tendency is more pronounced in a decision-maker who has a refined enough understanding of analysis to recognize that there are many situations in which it produces ambiguous or unsatisfactory answers, versus one who follows it more dogmatically.

4. The "blind spot" grows larger when a team is losing, or otherwise not performing to expectations. The explanatory vacuum does not just apply to isolated questions like pitch counts, but also to the all-important systematic one: team performance. There are times when an entire team may appear to be overperforming or underperforming reasonable expectations. Statistical analysis cannot usually reject the null hypothesis--that the variation could be explained by random variance alone--but it also cannot prove that something is not random. People are naturally inclined to seek out explanations in times of crisis, and decision-makers are naturally inclined to provide them, usually in the form of actions that appear decisive.

So here's what I think happened. I think Theo Epstein had some pretty specific and lofty expectations about how the Red Sox were going to perform this year. I think he was at something of a loss in explaining why they had failed to meet them. I think he felt that statistical analysis could neither explain it nor explain it away. I think that people important to Epstein were pressuring him for an answer. I think that Epstein was pressuring himself for an answer. I think that he and the rest of the Red Sox organization were too vested in Terry Francona to take the usual escape hatch and fire him. I think that, consistent with all accounts, Garciaparra was behaving like a colossal jerk, and the clubhouse environment was tense.

I think that Epstein put two and two together and got three-and-a-half, and made a bad trade.

Nate Silver is an author of Baseball Prospectus. 
Click here to see Nate's other articles. You can contact Nate by clicking here