August 9, 2010

Overthinking It

Judgment Qualls

by Ben Lindbergh

Printer-friendly

Contact Author

It's easy to construct a narrative around the Diamondbacks' deadline dump of Chad Qualls to the Rays for cash and a player to be named, especially if you're not overly concerned about accuracy. Picture this scene: At one end of the virtual negotiating table prowls Andrew Friedman, small in stature but flanked by his sabermetric death squad, a lethally efficient fusion of Wall Street and Revenge of the Nerds augmented by multiple BP alumni. In this distorted vision of reality, the Rays’ representatives likely sit down to deal after breakfasting on the marginal wins that they bought the night before with some loose change scrounged from under their couch cushions. On the opposite side of the room cowers Jerry Dipoto, the former player and current interim Arizona GM who has already managed to draw the ire of the sabermetric community by seeming to sell low on Dan Haren, subsequently compounding his PR problems by kowtowing to outmoded statistics. One could easily envision the showdown as a modern-day adaptation of the David and Goliath tale, in terms of the analytical acumen brought to bear by each side. Except this time, David dies.

To add additional drama to that Hollywood-style portrayal of events, the player whose fate hung on the outcome of the proceedings was a pitcher whom the best publicly available performance metrics deemed much more effective than the traditional statistics upon which Dipoto professed to rely. Judged by results alone, Qualls was the worst reliever in baseball before the deadline, recording an 8.29 ERA and blowing a quarter of his save opportunities (actually, the fact that a pitcher with an ERA so inflated blew only four out of 16 save chances bolsters the statistical argument for the "anyone can close" school of thought). On the other hand, both SIERA (3.79) and xFIP (3.85) were in almost perfect agreement that his skills remained far more intact than his ERA would indicate, a conclusion echoed elsewhere by Harry Pavlidis.

Given the parties involved, the stage may appear to have been set for highway robbery, but before we consider the trade itself, let’s take a look at the trajectory of Qualls’ performance from 2004 through the end of his time in Arizona:

Until this season, Qualls was roughly as reliable a commodity as one could count on in the volatile middle-relief market. From 2004-09, Qualls' ERA was confined to a range of less than one run, maxing out at 3.76 in 2006, and hitting a low of 2.81 in 2008. That stable ERA was no accident: From 2004 on, Qualls' SIERA ranged no lower than 2.80 and no higher than 3.93, even including his superficially abysmal work for Arizona this season. Only in 2010 do the two lines diverge dramatically, with his ERA skyrocketing while his SIERA barely budges.

Early last season, Steven Goldman wrote, "In truth, there are only a handful of relievers who are solid bets on more than a year by year basis: Mariano Rivera, Joe Nathan, Francisco Rodriguez, perhaps Scot Shields, and Chad Qualls. Once past that quintet, a team takes its chances." Of course, by the time Steven wrote that sentence, Shields' run of reliability had already come to an end, and Nathan hasn't pitched this season; even among the elite, pitchers are an undependable bunch. Still, the fact that Qualls merited a mention as part of such a select company provides some idea of his track record. His player comments over the years tell a similar story; in 2009, we wrote that Qualls was “an underrated commodity at this point, as he’s one of the hardest things to find: a consistent reliever,” and this past spring, while making the point that Qualls’ performance had barely wavered, we noted (with considerable foresight) that “it’s imperative to evaluate relievers on their ‘process’ more than the outcomes.”

So did the coldly calculating Friedman swindle the new “kid” (who’s almost a decade Friedman’s senior) on the trading block? The mid-season market for relievers was weak this year. Seven relievers found themselves involved in trades this July; of those, both Kerry Wood ($10.5 million) and Kyle Farnsworth ($4.5 million) were earning significantly more than Qualls, and Matt Capps, who wasn’t, brought a substantial return in prospects. None of the others available relievers could boast Qualls’ past success, though their recent records looked less ugly at a glance. Could Dipoto have extracted more from a team hoping to ride a Qualls rebound to October glory? Resident transaction analyst Christina Kahrl wasn’t impressed with Arizona’s return on any of its deadline deals, but it’s difficult to state definitively that Dipoto whiffed on a prime opportunity to reload in Qualls’ case, and not only because we don’t yet know the exact return.

Although we should weight the past several seasons more heavily than the fewer than 40 innings that composed Qualls’ season at the time of the trade, we can’t disregard that bloated ERA entirely. As Ricky Zanker noted, Qualls underwent knee surgery late last season, which could have subtly altered his mechanics, though whether that’s even partially responsible for his .427 BABIP in Arizona this season is anyone’s guess. A betting man would be wise to place his money on at least a partial return to form by the beleaguered reliever, but since we can’t entirely absolve Qualls of responsibility, that unsightly ERA does increase the risk associated with acquiring him.

Perhaps the real matter at issue here is how much one can realistically ask in return for a pitcher with an ERA over 8, even one with respectable peripherals. Let’s assume that both parties come to the table armed with the same information, fully aware that Qualls’ ERA this season is a poor reflection of his true talent. Even if the GM hoping to add Qualls to his roster values him as highly as he would had he had a lower ERA, can he overcome his sense of risk aversion enough to put his reputation on the line by extending an attractive offer? Can he sell management on the notion of sending money and prospects to a competitor for a pitcher whose season has been nothing short of disastrous, according to traditional measures? Can he sway the fans to his way of thinking, despite the fact that most of them don’t delve any deeper than ERA when evaluating pitchers? Even if rival GMs didn’t pretend to be more fazed by the exorbitant ERA than they actually were in order to extract concessions during the negotiations, Dipoto may have found himself in a tough situation thanks to the poor luck that appears to have plagued Qualls before the deadline.

In the wake of the Haren trade, Marc Normandin observed that Haren’s value was artificially lowered by a hitter-friendly park and a lack of defensive support. That’s undeniably true in a retrospective sense, but factors beyond a player’s control should affect his trade value only if potential traders and their trade partners fail to account for those factors properly. Leveraging the effects of park, defense, and luck may still lead to bargains in certain fantasy leagues, but it’s difficult to believe that suckers remain plentiful enough in major-league management that one could still use park effects or a fluky BABIP to put one over on an opposing GM and his assembled advisers.

After the league has caught up to speed in some respect, one wonders whether potential trade partners could get away with pretending to lag behind the rest of the class, selectively relying on hoary statistics when it suits their purposes. And, in such situations, would more openly enlightened teams be tempted to adopt the role of educator in an effort to avoid selling low, justifying their perception of their own trade baits’ worth by resorting to ad hoc summaries of sabermetric principles? I’m not suggesting that anyone is administering the last rites to market inefficiency, but at least a few such tricks of the trade must be on life support in 2010, rapidly giving way to ever more ingenious sleights of hand.

Any version of events that paints a picture of a clueless Dipoto having his pockets picked by Friedman's saber-savvy posse is likely overblown. For one thing, the negotiations took place at a distance, not in the fanciful tableaux described earlier; what’s more, they probably resembled something closer to a simple exchange of offers and counter-offers than the extended process of haggling, persuasion, and outright obstinacy that we associate with our own fantasy swaps.

Moreover, our snap impressions of front-office types, formed initially through sound bites and interview snippets, might not mirror reality as closely as we’d suspect. Dipoto's unapologetic encomium to wins, winning percentage, and All-Star selections in the aftermath of the Haren trade may have been nothing more than double-talk for the benefit of the assembled press, or a slightly more palatable alternative to owning up to blatant cost-cutting. If you've been keeping up with David Laurila's enlightening series of industry interviews, you know that Dipoto has at his disposal a support staff every bit as capable of appreciating defense-independent performance as Friedman's; at least one member of Arizona’s baseball operations staff doesn’t mind telling the world that he thinks that Qualls was a victim of poor luck (though one wonders whether a "European Scouting Supervisor" has any input on major-league roster decisions).

 Regardless of who “won” the deal (the Rays’ new reliever started out by giving them two scoreless innings before allowing three runs in his third, while Arizona’s player remains nameless), the Qualls trade is an interesting litmus test for one’s faith in the ability of advanced metrics to predict future performance. That a team in the thick of a pennant race would go out of its way to acquire a player whose worth this season has been captured most clearly by statistics that weren’t widely available (or in some cases, weren’t available at all) until fairly recently might serve as an indication of the prominent place that the statistical side of player evaluation has attained in today’s game. After all, we know that the Rays didn’t have an advance scout on the scene.  

Ben Lindbergh is an author of Baseball Prospectus. Follow @benlindbergh
Click here to see Ben's other articles. You can contact Ben by clicking here