Divide and Conquer, AL... (06/03)
25 comments have been left for this article. BP Comment Quick Links
| |
|
|
June 3, 2011 Baseball ProGUESTusCan Baseball Expertise Be a Bad Thing?Believe it or not, most of our writers didn't enter the world sporting an @baseballprospectus.com address; with a few exceptions, they started out somewhere else. In an effort to up your reading pleasure while tipping our caps to some of the most illuminating work being done elsewhere on the internet, we'll be yielding the stage once a week to the best and brightest baseball writers, researchers and thinkers from outside of the BP umbrella. If you'd like to nominate a guest contributor (including yourself), please drop us a line. Sam Miller writes about baseball for the Orange County Register. He covered local government, education and autism before moving to the sports section in 2009. He lives in Long Beach with his wife and newborn daughter, and you can follow him on Twitter at @sammillerocr. 1. Before the 1993 season began, Mike Scioscia was talking to a Los Angeles Times reporter, reminiscing about his 11 seasons with the Dodgers. Suddenly, he stopped. "I hope you are not writing this article like I'm dying," he said. "My career isn't over.” The Dodgers disagreed. They had just let Scioscia leave as a 33-year-old free agent. He had already lost playing time the previous season to rookie Mike Piazza, who was called up in September and started 16 of the team’s final 31 games. Scioscia’s final at-bat as a Dodger came as a pinch-hitter with his team trailing 6-1. "Other guys who, in the near past, have had bad years or been injured have always been given the opportunity by the Dodgers to come back and have a good year," Scioscia said. "I feel like last year is the first bad year I've had in a long time.” By one measure—not a measure anybody cared about at the time, maybe a spurious measure, but a measure—it wasn’t even a bad year. The Dodgers’ pitchers had a 3.16 ERA when he caught, and a 3.78 ERA when Piazza or Carlos Hernandez caught. That was consistent with what the Dodgers got almost every year. From 1984 through 1992, Scioscia’s final nine years, the Dodgers allowed 3.71 runs per nine innings when he was catching, and 4.00 runs per nine innings when somebody else was catching. (On a pitcher-by-pitcher basis, he was generally better, too.) "I don't think it was a coincidence that the staff was number one or number two in earned-run-average when Mike Scioscia was here,” Orel Hershiser said at the time. “He's the one behind the plate calling the pitch selection. You can't have 10 pitchers and say they are all smart. Human nature says that all 10 aren't smart." Imagine a world where Catcher ERA caught on. In that world, Mike Scioscia could have claimed credit for somewhere between 200 and 300 runs saved in his career. In that world, Scioscia would have some people debating his Hall of Fame case. In that world, the Dodgers might not have replaced him with a slugging catcher who couldn’t catch, and in that world Jeff Mathis would make sense. 2. Shortly after I began writing about the Angels for the Orange County Register—and before I’d ever met Scioscia—I wrote a post about the reported “competition” in Spring Training for the starting catching job. This was in 2009, the year after Napoli’s .960 OPS. “A bit of springtime competition could be good for both of them,” I wrote, “but there’s no way Mike Napoli and Jeff Mathis are really fighting over that catcher’s job. This is just for show, folks.” I was dead wrong. Mathis started Opening Day, and the next day, and 76 other times that season. In the four years they shared the catching job, Napoli started 282 times, Mathis 231. If Mathis were to double in each of his next 100 at bats, his career slash line—.269/.314/.455—would still come up well short of Napoli’s .251/.346/.485 line as an Angel. Mike Scioscia isn’t a stathead kind of guy. He’s not as anti-stat as you might imagine—he’s extremely wary of sample-size issues, he doesn’t believe in distinguishing between earned and unearned runs, and he thinks pitcher wins are a bad way of measuring starters—but he is extremely skeptical of advanced defensive stats. He says they don’t account for the role of advanced scouting, positioning and, yes, even the catcher’s role in calling pitches that reflect the scouting and positioning. There’s only one defensive statistic that he thinks can accurately reflect the player’s role: Catcher Runs Allowed. “An absolute tool as to how a catcher relates to a pitcher’s performance,” he called it. “Let me put it to you this way,” he once said. “If you string out 162 games and you have one catcher who is giving up one run a game less when he catches, on the net runs end of it, he’s 162 runs ahead, right? So the other catcher has to produce 162 runs more than the other guy just to break even. I think a catcher is going to influence a game and a season behind the plate more than he is with his four at-bats a night.” Perfect: the one advanced metric he approves of is one that had essentially been considered disproven for a decade. And the one advanced metric he approves of is the one that conveniently makes him look like he was a superstar. 3. All of this goes to the very nature of expertise. Mike Scioscia is an expert at catching. He knows more than I know about catching, and more about his own catchers, and more about his own decision-making process. Do you know how absurd it feels when I’m standing in his office arguing with him about Jeff Mathis’ catching skills? But at the core of “expertise” is a dangerous belief in oneself. The more of an expert you are, experts (ironically!) tell us, the less you consider the horrifying fallibility of your own brain. Michael Lewis noted this in Moneyball: “There was the tendency of everyone who played the game to overgeneralize wildly from his own experience.” The failure of expertise is a theme of nearly every pop-science best-seller from the past decade—Malcolm Gladwell and The Wisdom of Crowds and Jonah Lehrer. (Wine experts can’t even tell the difference between red wine and white, when food coloring is added!) “And the more knowledge that expert has, the worse, interestingly, it becomes, because the expert is using the knowledge very selectively to justify increasingly extreme predictions,” said Philip Tetlock, the author of Expert Political Judgement. Would Jeff Mathis still be in the majors if Mike Scioscia had been a third baseman? Would the Angels be better off? 4. Each year has made it harder to say he’s definitely wrong. Maybe he’s been right all along. The first time I looked at the two catchers’ “performance,” I found that Mathis outperformed Napoli in 2008 by runs allowed, but that the difference was almost entirely related to batting average allowed on balls in play. Try to come up with a flukier-sounding stat than Catcher BABIP. I dismissed it. Then in 2009, five Angels starters pitched at least 100 innings. Mathis’ CERA was at least a half-run better for each of the five, his K:BB rates were better with four of the pitchers (and essentially tied with the fifth), and his BABIPs were again significantly better for four of the pitchers (and essentially tied with the fifth). In 2010, five Angels starters pitched at least 100 innings. Mathis’ CERA was significantly better for four of them, and he again trounced Napoli in K:BB and BABIP. There were differences in the way they called the game: Mathis appears more likely than Napoli to call off-speed pitches, especially in traditional fastball counts. And Mathis “earns” an extra strike call every two games or so. If switching a ball to a strike is worth .161 runs, as Dan Turkenkopf found, Mathis has an edge of about 10 runs over a full season, by pitch-framing alone. Meanwhile, the CERA numbers Scioscia was citing kept piling up. And meanwhile, I kept mocking his decision to play Mathis. So now which of us was using knowledge selectively to justify increasingly extreme predictions? 5. There’s so much noise, though. I wondered whether there were other variables that correlated as strongly with the Angels’ run prevention during the Mathis/Napoli years. I tried one variable and found nothing. I tried a different variable, and it hit: 2006:
With X: Angels allow 4.06 runs per games 2007:
With X: Angels allow 3.76 runs per game 2008:
With X: Angels allow 3.25 runs per game
2009: 2010:
With X: Angels allow 4.35 runs per game As clear as can be: X correlates to improved performance by Angels pitchers and defenders, even more (by a little) than Mathis starts vs. Napoli starts. Variable X, incidentally, is “Phillies Scored Two Or Fewer Runs The Same Day.” I’m pretty sure there’s no causal link. 6. A Brief History Of How The Prevailing Literature Rejects or Supports Mike Scioscia’s Perceived Genius/Idiocy:
1989: Craig Wright, Inner Circle Sabermetric Hall of Famer, writes in The Diamond Appraised: “I fell in love with (CERA) the minute I read it.” He uses it to evaluate catchers.
1999: Keith Woolner writes for BP, “Catchers do not have significant differences among their game-calling abilities.” Wonders whether this means Frank Thomas should catch.
2002: Bill James writes that there may be significant run-suppressing ability for a catcher, but that it’s likely undetectable.
Later 2002: Woolner says any run-suppressing ability would be undetectable if it existed, but might not exist anyway.
2003: Chris Dial takes a DIPS approach to Woolner’s 1998 research and agrees with the conclusion.
2008: PITCHf/x comes along. Dan Turkenkopf suggests that framing ability alone might be worth 25 wins per year, concedes this sounds crazy, but “maybe we’ve been wrong the whole time.”
2010: Bill Letson suggests a 13-win range between the best and worst catchers based on pitch-framing.
Later in 2010: Sean Smith writes that “catchers have a significant impact on the performance of the pitchers they catch.” Not significant enough to justify Mathis starting over Napoli.
2011: Mike Fast puts the range at around +/- 20 runs per year based on pitch-framing alone. Because this is Baseball Prospectus, here is a gratuitous chart:
Average smartness: 5.5. 7. Scioscia doesn’t actually use Catcher Runs Allowed the way you probably think he does. He’s not interested in each catcher’s overall numbers, for instance, but specifically how well each does with each pitcher. He says he doesn’t use even those numbers to make his decisions about playing time, but as a starting point to look for what is causing the differences. He won’t talk about what those causes are, and some of them are immeasurable. For instance, the Angels worked with Napoli on setting a target. That seems like a small detail, but the Angels don’t feel that way—they think it has a real effect on pitch execution. Scioscia originally learned about Catcher Runs Allowed when he was a coach in the Dodgers’ minor-league system, where it was used not just to assess but to motivate. The Dodgers believed that if young catchers knew they would be evaluated based on this statistic, they would be more focused on figuring out how to stop runs from scoring by any means possible. “Everyone has a philosophy they view the game through. Our philosophy is that the catcher-pitcher relationship is in play for every pitch of the game,” he says, and none of that sounds unreasonable. I tell him that I actually trust his expertise, and if he told me that he was starting Mathis based on his eyes and experience I might even believe it could be the right call. It’s his use of the particular stat that bothers me. “When you see it with your eyes and go back and check it,” he says, “I don’t know if there’s one case where it’s been wrong.” 8. We don’t live in that imaginary world. Catcher ERA didn’t catch on, not really, and nobody talks about Mike Scioscia’s Hall of Fame case. But if he wins another World Series, maybe two, then we will someday debate his case as a manager. This one decision—Mathis vs. Napoli—might be the defining detail people remember about him. I don’t know where we’ll be when that happens. Right now, the literature suggests he is more likely wrong than right, but also more likely a supergenius than a total idiot. There will likely be some day in the future when we all think Mike Scioscia was right. Then there will be another day in the future when we all think Mike Scioscia was wrong, and so on.
Sam Miller is an author of Baseball Prospectus. Follow @SamMillerBB
|
Fantasy Beat: Weekly Planner #10
Is catcher evaluation statistically impossible to achieve? How do you combine all the things that catchers do -- framing the ball, calling pitches, blocking balls in the dirt, throwing out players attempting to steal, intimidating runners into not even trying to steal, blocking the plate, calming and refocusing pitchers, etc. -- into a defensive metric? Just because a catcher calls a pitch does not guarantee the pitcher will throw it well or to where he places the glove. And the pitcher's ability to hold a runner affects a catcher's caught stealing stats. He's the most important defensive player (pitchers change while the catcher remains) yet may be impossible to measure. Oh, and that's before calculating offense...