October 14, 2013

Baseball Therapy

Does Postseason Experience Really Matter?

by Russell A. Carleton

David Ortiz has come to the plate in a playoff game more than 300 times over the course of his career. Justin Verlander has started 14 games in the postseason and faced more than 300 batters. When they meet during this week’s American League Championship Series, who has the advantage? What about when Ortiz inevitably faces Tigers lefty reliever extraordinaire Drew Smyly, who prior to this year had a postseason resume consisting of four innings (all last year)? What about postseason neophyte (and rookie in general) Jose Alvarez? Does Verlander’s experience in the postseason give him an advantage over Will Middlebrooks, who made his playoff debut last week, that he doesn’t have against Ortiz, who’s been doing this since 2002 with the Twins? (Post hoc: I wrote this before Big Papi hit his big home run. #YCPB)

Of course it does! Haven’t you listened to former players, TV pundits, newspaper columnists, and everyone on sports talk radio? The playoffs are a “different game.” The pressure is “turned up a notch.” Guys who have been around know how to find “an extra gear.” Veterans are better at handling the “pressure cooker.” Yes, I’m writing an entire paragraph of sentences with “trite sports clichés in mocking quotation marks.”

Like a lot of conventional baseball wisdom, the idea of previous playoff experience conferring an advantage on a player (or a team) smacks of amateurish pop psychology that’s been repeated enough times for people to stop questioning whether or not it’s true. (Well, that’s not entirely true. In 2008, David Gassko found evidence that teams that had more playoff experience were more likely to win their series than we might expect. Also, Joe Sheehan gave it the idea the Joe Sheehan treatment five years ago.) In the past, we’ve seen cases where conventional wisdom has proven to be true, and others where it has proven to be taurine feces. I vote that we take a look at what the numbers really say.

Two weeks ago, I looked at whether postseason performance was related to whether a team had to fight down the stretch to get into the playoffs, as opposed to cruising through September already assured of their postseason berth. It turned out that this sort of trial by fire had some (statistically iffy, but still worth mentioning) positive effects for a team’s pitchers. I argued that this might be a case of stress inoculation, where players get used to the stress of playing high-leverage games before playing…more high-leverage games. Might the same sort of thing be happening here? Is a player who has faced down the pressure of the playoffs before better able to handle it this year?

Warning! Gory Mathematical Details Ahead!
I looked at all postseason games from 1995 (the beginning of the Wild Card Era) through 2012. For each plate appearance, I used my now-standard log-odds ratio method to code for the chances that the PA would end in one of seven basic outcomes (K, BB, HBP, 1B, 2B/3B, HR, out in play) based on the regular-season stats of the pitcher and hitter involved (min 250 PA or BF). As always, this served as a control for the overall talent level on the field. As per usual, I ran a series of logit regressions to see what would happen when I added in some extra variables.

For each plate appearance, I determined the number of previous plate appearances by each batter in his personal postseason history. I did similarly for pitchers. I also determined how many previous games both the batter and pitcher had appeared in (or more accurately, had either pitched or hit in). For the nitpickers, I included games and plate appearances that happened before 1995 in these calculations. For example, the 1995 Cleveland Indians included playoff veterans Orel Hershiser, Dennis Martinez, Eddie Murray, and erm, Paul Sorrento. My regressions were aware of their previous exploits.

When I entered the number of previous plate appearances as the predictor, I got a couple of isolated, marginally significant (p-values were hanging out in the low teens) results, and even if we pretend that they were significant, they had some very modest effect sizes. (A lot of experience—say, 30 prior games—might move the needle by a couple of tenths of a percentage point away from expectations in either direction). Hitters with a little more experience were slightly less likely to strike out but also less likely to hit home runs, and pitchers who had been around the block a few times allowed slightly more singles and fewer extra-base hits. When you denominate things by games played, there are some significant findings, but none of them are the same. Usually, if there’s a real pattern, you see similar findings across similar variables. This is not the case here. I later split the sample into plate appearances against the game’s starting pitcher vs. a reliever. The same basic non-findings fell out.

I tried coding for whether or not this was the hitter’s (or pitcher’s) first postseason plate appearance. I also looked to see whether it was his first game. Maybe there’s something about having to shake the jitters out the first time, but then you go back to normal. There was no evidence that first plate appearances were any different than the rest of them for batters or pitchers.

I also looked at whether it was the pitcher (or batter’s) first year in the postseason. Nothing significant shook out there. When I restricted the sample to the division series only (because if the nerves are gonna hit, it’s going to be early in the postseason), there were still no effects for being a first-timer.

There’s just nothing here to talk about. There is no evidence that postseason experience (and I attempted five different definitions of “experience”) has any effect on players in the postseason over and above their previously established talent levels. The idea that postseason experience confers some sort of advantage on a player or team is not supported by the data. If it were true, we would see some sort of departure from what we would otherwise expect based on regular-season stats. It’s not there.

Trading Jimi Hendrix for Simon and Garfunkel
If there is no effect of previous postseason experience on results, why does the narrative survive? It’s not as though there haven’t been recent examples of teams low on experience (the 2003 Marlins, the 2005 White Sox, the 2007 Rockies, and the 2008 Rays) that have made World Series appearances despite not having appeared in the playoffs in the near past. For too long, the only question has been “Are you experienced?” All I’d suggest is that a man hears what he wants to hear and disregards the rest.

Let’s first talk about why “postseason experience” never made sense as something that would make much difference in outcomes, once we get away from overly-simplistic pop psychology. The most common argument for why experience should matter is that experienced players will know how to handle the increased pressure of the postseason environment. I buy that the postseason really feels more stressful to players. I’d even buy that this might affect a player’s performance if not handled correctly. The big hole in the theory has to do with understanding how people cope with stress.

Everyone has a way to deal with stressful situations. Some are effective, some are destructive, some are silly, and some are all three at the same time. Looking at my own life, I use humor, nervous pacing, and running statistical analyses on baseball as ways to deal with whatever’s bothering me. I have for a long time. You might have a similar style or do something totally different, but you have something. Now suppose that I put you in a situation that you had never been in, and that you found stressful. For example, when my first daughter was born, I had no direct experience to guide me on what was going to happen. My wife was annoyed that I was calculating WARP values in the delivery room, but she’s since forgiven me. It wasn’t the first time I’d coped with a stressful experience, and I used techniques that had worked in other circumstances. Coping strategies are supposed to be flexible like that. If they weren’t, every time you encountered a new stressor, you’d have to come up with a new way to handle it. Humans would not have survived if this was the case.

It’s possible that some players have very poorly formed coping strategies, in which case the problem is lack of a good coping strategy, not lack of postseason experience. It’s possible that other players have well-formed coping strategies, and that one of the previous stressors with which they have dealt is the stress of the postseason. But the argument that postseason experience is the only way to learn these strategies pre-supposes that a man can’t possibly apply lessons learned from other parts of life to the stress of Game 1 (or Game 7). Baseball isn’t that special. If there’s a common thread behind a lot of pop psychology misperceptions of baseball, it’s that we seem to forget that these guys do other things in life than play baseball. If you want to make a half-hearted justification of postseason experience being meaningful, you can at least guarantee one time in a player’s past in which he’s had to use stress management techniques. And that’s nice. It’s just that people deal with stress of all kinds, even if it’s just first-world problems like, “Oh dear, I’m being paid several million dollars to be on national TV and we might lose this game.”

Then there’s the argument that while the experience of stress cuts across all parts of life, the amount of stress that a player will feel will be a great deal more than he has ever felt. Ever. It’s a seductive argument until you think about whether or not the most stressful events in your own life were related to your job. They might have been, but that’s probably not the case for everyone. Even if we agree that playoff baseball is the most stressful thing that a player will face in his life, there’s something about baseball that’s different from most work-related, stress-inducing “big events.”

When I think of big events at work that make me a little nervous, they revolve around things that are (to borrow a term from sabermetrics!) high leverage and that I don’t do very often. For example, in my line of work, I sometimes travel to professional conferences and present data. I’m a mess an hour beforehand in my hotel room. One thing that can help someone calm down before an event is to go about as much of one’s normal routine as possible and try to practice doing whatever it is so that when the big moment comes, that action is part of muscle memory like everything else. Baseball players have the luxury of the fact that the thing that they have to do to get through this situation is to play a baseball game. And if there’s one thing that baseball players have a lot of experience in and practice a lot, it would be, well, playing a baseball game. There’s no doubt that the game is high leverage, but the actions he needs to perform are things for which he has a well-defined script. That’s going to be protective.

So, yes. I believe that baseball players do feel the pressure of the playoffs. I would guess that it can affect performance on the field. My work from two weeks ago on momentum heading into the playoffs suggests that mindset can play a role in performance, particularly in the postseason. I just don’t believe that the only way that a player can learn to deal with the pressure is by previous postseason experience. I can understand why this idea might stick around. It’s the kind of theory that makes sense if you don’t think about it too hard. It’s also easy to look up who’s been in the postseason before, but harder to do 25 in-depth coping strategy interviews with players.

I can imagine that there are some readers out there who are nearing the end of this article and are forming sentences in their heads that begin with “Yeah, but there’s no substitute for experience…” I would kindly point out that the numbers would suggest otherwise. I would also suggest that the important thing isn’t having experienced the playoffs yourself, but having some way to keep level. About the best case that you can make for the importance of playoff experience is to say that a guy who’s been around (and to a few postseasons) might be able to teach some coping skills to the younger players who haven’t been around as long. And maybe a few of them haven’t really yet developed those skills.

But make no mistake. The numbers are very clear here. This week, when you hear someone extol the virtues of playoff experience, you’ll know better. Prior playoff experience has no predictive power over outcomes in the postseason. I would gladly tell you otherwise if it were true, but the numbers just don’t back that up.

Russell A. Carleton is an author of Baseball Prospectus. Follow @pizzacutter4
Click here to see Russell's other articles. You can contact Russell by clicking here