March 14, 2013

Overthinking It

15 Questions I've Been Asking Myself Since the SABR Conference

by Ben Lindbergh

Printer-friendly

Contact Author

There were, by my count, 25 talks, panels, or presentations at last week’s SABR Analytics Conference in Phoenix. I couldn’t attend all of them, since some overlapped, but I made it to as many as possible. I’ve already written about the most interesting thing I heard, but the Indians’ sabermetric approach to marketing was just one of many intriguing topics that made me start scribbling notes during the three days I spent listening to smart people talk about baseball. (Many of those topics were brought up by Bill James, which probably isn’t surprising.)

Below I’ve listed some of the questions asked (either explicitly or indirectly) at SABR that are still on my mind a week after the conference began. I don’t have the answers to all of them, but that’s okay, because, as James said, “The key is to find the questions.” (Note: Only a few of the events are available online, there was no convenient place to put a computer, and I scribble only so fast, so I may have mixed up a detail or two.)

Should teams pamper their players?
Brian Kenny was the first featured speaker, and he spent much of his time talking about competitive advantages that he thinks teams still aren’t exploiting. He cited an example from Soccernomics of one team that had an entire department devoted to treating its high-priced transfers like royalty, essentially offering them full concierge service from the moment they signed. According to Kenny,

They found that most of these players that would come in and transfer for big money, they’d be busts. And they’d be busts because they were uncomfortable, their family was uncomfortable, the change was more difficult than they thought. They were ignoring the psychological impact on that player. And one team in particular said, ‘We are going to make sure all of our guys are as happy as they possibly can be,’ and the results were tremendous. I think that’s the next wave.

It’s not as if most major leaguers have it hard, but it’s reasonable to suggest that a team could distinguish itself by offering players more and more off-the-field perks, simultaneously optimizing its current players’ performance and making itself more attractive to potential acquisitions.

Bill James wasn’t buying it. Speaking the next day, James admitted that he’d had the same thought himself, and that what Kenny had proposed was literally true: “If you do more to make sure your employees are comfortable, you’ll get better results.” But according to James, any advantage derived would be short-lived. James believes that happiness adheres to the “Law of Competitive Balance” he formulated in the 1983 Abstract, which states that certain forces tend to conspire against strong teams and in favor of weak teams in ways that reduce the difference between the two. He cited the findings from this study, which were summarized in a Forbes piece last November:

The study found that the overall happiness levels of lottery winners spiked when they won, but returned to pre-winning levels after just a few months. In terms of overall happiness, the lottery winners were not significantly happier than the non-winners. The accident victims were slightly less happy, but not by much. The study showed that most people have a set level of happiness and that even after life-changing events, people tend to return to that set point.

James concluded, “The same happens in a baseball team. If you try to make players happy all the time, what you wind up with is spoiled players who can’t do anything for themselves, and they’re not any happier than they would be. You just move the standards.”

In other words, pampering a player might make him feel more at ease in the short term, but it wouldn’t make him happier down the road. And it would mean more time and expense for the team. 

Will the knuckleball ever be banned?
Brian Kenny believes that one day the knuckleball could be outlawed, like the spitball. Knuckleballers, Kenny says, are potentially too effective: they can throw more innings, work on shorter rest, and remain productive at more advanced ages than most pitchers. Up until now, teams have been willing to wait for the occasional knuckleballer to come along, but Kenny sees an opportunity for some team to “open up an academy and start churning out these guys.” If that were to happen, he suggests, knuckleballers could proliferate to a dangerous degree, endangering the tenuous pitcher-batter balance and forcing MLB to take action.

Of course, if the knuckleball has the potential to be effective enough to upset baseball’s competitive balance, why hasn’t some team made more of an effort to develop knuckleball pitchers already? Kenny believes it’s just a case of bias without basis. “There’s a conditioning, a bias against valuing these guys, for no reason except it’s, what, unmanly? It’s a gimmick pitch?” he asked at SABR. “It’s nonsense. What does it actually do? It gets guys out.”

If you’d asked me why the knuckleball hasn’t become more common, I probably would’ve said “because it’s a hard pitch to throw.” Perhaps there’s something to that. But Bill James revealed another, equally persuasive reason I’d never considered. And he was speaking from experience:

“Theo [Epstein], when he was in Boston, was always trying to develop knuckleballers. He always had some plan about how he was going to develop knuckleballers, and it never worked. And it never worked because there’s a long, long series of small barriers to it that are invisible from the outside.”

Those barriers to knuckleballer development are simple, but significant: there’s no one to coach them and no one to catch them. Since there are so few former knuckleballers, there are few coaches with specialized knowledge of the pitch who can nurture the next generation of knucklers. And since there are so few people who throw the pitch, there are just as few who can catch it. As a result, it’s hard for a knuckleballer to throw a side session unless he wants to retrieve the ball himself. And he can’t be brought in with runners on base, because those runners will score on passed balls. As James concluded, “It’s just very, very hard to develop knuckleballers for reasons that are not really apparent until you try it.” You can see some of that subtext in Jonathan Zeller's recent BP profile of Red Sox minor-league knuckleballer Steven Wright.

A knuckleball academy might solve some of those problems by being equipped with capable coaching and catching staffs, but eventually the knuckleballers would still have to face professional hitters, who probably wouldn’t be willing to enroll in the academy and take BP against knuckleball cadets when they could be advancing their own careers. And when the time to face real hitters in actual games arrived, the same catching problems could crop up. There’s also the possibility that hitters could become accustomed to the knuckleball if they saw it more often, which would make the pitch less effective. 

Are high draft picks less likely to sign early extensions?
This is one question that research could settle. It was cited during the player development panel as a potential downside to drafting early. Players sign team-friendly extensions before they become free agents to ensure their families’ financial security in the event of an injury. Therefore, the thinking goes, players who were picked early and got big bonuses might have less incentive to sign than late-rounders who outperformed their draft position but have very little tucked away in the bank.

Of course, Evan Longoria, the patron saint of players who sign team-friendly extensions, was a third overall pick who got a $3 million bonus from Tampa Bay. To put that into perspective, Bloomberg Businessweek reported late last year that the average American with a bachelor’s degree makes $2.4 million in his lifetime. So before Longoria had played a professional inning, he’d already earned more than the average college-educated American ever will. That's financial security for you. And he never even got his degree!

Could the minor leagues be better structured?
James suggested that the structure of the minor leagues doesn’t lend itself to efficient player development, observing that a “system that develops when each individual follows his own selfish needs is often an inefficient system.” He believes that “from the standpoint of the game as a whole, there would be huge advantages to restructuring the minor leagues so that players were not sorted as belonging to teams until they reached a much higher level.” In such a system, the draft would be held when players were a year or two away from the majors; before that, they wouldn’t be attached to any particular team. One possible side effect of that system: it might also reduce the importance of scouting, since teams would have more reliable performance data with which to compare players before making their picks.

On what planet(s) did Ichiro Suzuki and Mariano Rivera originate?
At this point, SETI should probably just call off the search: Ichiro and Rivera are the most unusual life forms we’re ever likely to encounter. Sportvision’s Graham Goldbeck, who (presumably) gets paid to play with FIELDf/x, HITf/x, and COMMANDf/x all day, gave a presentation titled “Batted Ball Success by Depth in the Zone.” Matt Eddy has a thorough write-up of the whole thing here, but I want to focus on what Goldbeck revealed about the two future Hall of Famers who regularly do things no one else does.

HITf/x tracks the precise points in space at which batters make contact, and Goldbeck had data on over 600,000 balls in play since 2008 to examine (which, again, he probably got paid for). He found that batters who make contact with pitches well in front of the plate (intuitively) tend to pull the ball and hit for more power, while those who wait (or react more slowly) and hit the ball when it’s closer to the catcher tend to go the other way and hit for less power. The average point of contact is about four inches in front of home plate, but max ISO on contact comes about eight inches earlier.

Goldbeck also found that contact points tend to stabilize quickly—according to Eddy, after about 25 balls in play. Except for Ichiro’s. Ichiro’s contact point was the least stable of any batter’s: he hits pitches in front of the plate, deep in the zone, and everywhere in between. We already knew Ichiro had crazy bat control, but it’s nice to see Sportvision’s newfangled stats confirm it. I’ve been skeptical of claims that Ichiro could hit for much more power if he simply decided to try, but I’m a little less skeptical now.

And then there’s Rivera. Goldbeck found that pitchers who induce contact deep in the zone tend to be hard throwers (which makes sense, since it’s harder to catch up to their pitches). There were three notable exceptions to this general rule—relatively slow throwers who still induce deep contact. Two of them were Brad Ziegler and Randy Choate, specialist relievers with weird arm angles whom hitters can’t get a good read on. The third was Mariano Rivera, who throws the same damn pitch every time.

When the two went head to head (which, sadly, we’ll almost certainly never see again), Ichiro’s weirdness won out: he hit .400/.438/.667 in 16 plate appearances against Rivera.

How big is baseball data going to get?
On Saturday, SABR president Vince Gennaro provided some figures about the exponential increase of baseball data over the last decade. He estimated that when Moneyball was published in 2003, the space required to store all statistical data on major-league games produced to that point–mostly box scores and some play-by-play logs—came to less than 1.5 gigabytes. The last five years have produced six percent of all big-league games ever played, but over 95 percent of the data about them (without counting minor-league data or fledgling FIELDf/x), since we’re now storing so much information about players’ processes as well as the outcomes of each play. All in all, we’re up to roughly 30-40 gigabytes, with many more to come.

Gennaro stressed that we need both new methods and new hardware to process data on that scale. He mentioned that YarcData, a division of Cray (“the supercomputer company”) that constructs and runs complex queries like the ones that produced the “pitcher clusters” he presented, uses computers with eight terabytes of RAM. The maximum amount of RAM the little laptop I’m using to type this can handle, assuming I’m willing to void the warranty, is eight gigabytes. It freezes for five minutes when I click on long email threads. Cray’s computers can run a query in a couple of days that might take a year for most companies to complete, or calculate the Answer to the Ultimate Question of Life, the Universe, and Everything while Deep Thought is still stuck at the sign-in screen. That’s the kind of processing power that some baseball teams might be about to have at their disposal.

One other tidbit from Gennaro’s talk: when he analyzed pitcher clusters based on a whole host of process characteristics, he noticed that pitchers from the same organization were grouped together more often than he expected, which suggests that we might be underrating the extent to which coaching and organizational philosophies affect pitchers’ approaches.

Do situational relievers make sense?
Recently, Bill James wrote, “Right or wrong, it is my opinion, until somebody can show me where I’m wrong, that carrying left-handers in the bullpen is a complete waste of time and resources.” At SABR, he said he doubts that the compulsive pursuit of the pitching platoon advantage is worth more than 5-8 runs to most teams, much fewer than he thinks those teams could gain by expanding their benches, platooning position players, and using defensive replacements, pinch-hitters, and pinch-runners aggressively.

Someone asked James why the Red Sox still use what he believes to be a counterproductive strategy, and he explained that change isn’t as simple as voicing an unsupported assertion and expecting John Farrell to adopt it immediately. A couple factors James might not be taking into account: first, the deterrent value of situational relievers. If a team knows its opponent has no southpaw in the pen, it can stack its lineup with lefties whenever a righty starts or a lefty starter leaves the game. And then there’s the effect of forcing one’s remaining bullpen guys to pitch in longer bursts, which would likely make them all a little less effective without the right mix of relievers. Some helplessness at the hands of a big lefty bat in the late innings wouldn’t be the only downside to doing away with LOOGYs.

Are we smarter about baseball than we used to be?
The obvious answer is “yes,” since we certainly know things now that we didn’t before. But James isn’t so sure. He believes that new fallacies inevitably arise to replace the old ones, and that “there will never be a shortage of ignorance—we’re just doing different stupid things.” To hear him tell it, the team he works for is as guilty of that as any. About the Red Sox’ recent free agent signings, James said, “Some of them were terrible moves at the time we made them, and we should’ve known better.”

Is the public “ahead,” or are front offices?
Teams have armies of scouts and gigabytes of proprietary data at their disposal. But James believes that the public sector will always be ahead, because of its greater accumulated brainpower. According to James, team analytics departments are to public-sector statheads what the White House is to the press. The press is always complaining that the White House is slow to respond to various issues, but it’s not because of incompetence; it’s because the White House doesn’t have the same information-gathering abilities as the combined power of the press corps.

I see his point. But still: all those scouts, and all those proprietary stats.

What weight should we assign to “scouts” and “stats”?
Okay, so we all agree that both scouting and statistical data are important, and we know that smart teams (read: all teams) synthesize the two. But does that mean we should assign the same weight to each one, or is there a better breakdown?

Dodgers President and part-owner Stan Kasten said he favors scouts 60-40, in a general sense. When it comes to in-game tactics, he’s fully in favor of stats, since a run expectancy table can tell you what happened in thousands of identical situations that arose in the past. But he believes that while stats are great at telling us what has happened—in that respect, in fact, he acknowledges they can be close to perfect—they’re not as useful in predicting the future. Kasten said that prospects aren’t like base-out situations, since no two of them are exactly alike, so you can’t make player development decisions like you make in-game moves.

One of Kasten’s biggest laugh lines was something a respected rival GM told him: that he really likes analytics when they agree with what the scouts say. According to Kasten, that’s the case 80 to 90 percent of the time.

What is clubhouse chemistry worth?
I mentioned this on Tuesday’s podcast, but during the player panel, Brandon McCarthy, one of the game’s most statistically savvy players, said that without Brandon Inge and Jonny Gomes—two players with a combined 2.8 WARP in Oakland—the 94-win 2012 A’s might have been a 70-win team. Nick Piecoro later asked him to elaborate, and he did. I doubt McCarthy actually believes the effect was anywhere near that large; more likely he was just trying to counter the chemistry deniers with some hyperbole of his own.

It’s difficult to quantify team chemistry—we’ve tried—but it probably doesn’t get us any closer to the answer when either “side” of the issue exaggerates its estimate of the impact for effect. It seems safe to say that it’s not worth 20 wins—if teams believed that it were, Inge and Gomes would be making much more money—and it’s not worth nothing. Can’t we all agree that the true effect is somewhere between the extremes? If so, we can stop squabbling and start figuring out how to make our estimates more precise.

Are steroids still a problem?
James said “steroids are gone,” though he later clarified that he thinks fewer than 1 percent of players are still using them. He wasn’t including testosterone or HGH in that estimate, but he’s impressed with the measures Major League Baseball has taken to eradicate the use of those substances. PEDs might still be a problem in a retrospective sense, and they’ll continue to pop up as long as players from the "Steroid Era" are on the Hall of Fame ballot, but if James is to be believed, their impact on baseball today is negligible.

How much does a pitcher’s extension matter?
Unlike PITCHf/x, Trackman, a radar-based ball-tracking system, records a pitcher’s actual release point. The closer that release point is to home plate, the less reaction time the hitter has, and the higher the perceived velocity of the pitch. According to Alan Nathan’s SABR presentation, one foot of extension equates to about 1.5 miles per hour of perceived velocity. Fourteen big-league teams now have Trackman installed in their stadiums.

Naturally, taller pitchers tend to get more extension, but size isn’t all that matters—so does stride length, among other things, which explains how David Robertson, who stands 5’11”, can have better extension (and thus a “sneakier” fastball) than much more physically imposing pitchers. Teams are paying attention to pitcher extension, and so should we, to the extent that we can without access to the data.

Should front offices play a larger role in dictating in-game moves?
Brian Kenny thinks we’re getting to the point at which teams will have a stat-savvy assistant GM in the clubhouse or the dugout, where he’ll feed the manager information and offer advice on in-game moves. But Jed Hoyer says it’s the front office’s job to filter information for the manager, not to meddle with his moves: if players perceive that the front office is pulling the strings, it undermines the manager. On a separate panel, James echoed Hoyer’s concern, saying, “You don’t put a front office man in a suit in daily contact with the players.” One potential solution: more analytically inclined bench coaches, something James thinks we’re already seeing.

After the hiring of Walt Weiss and Mike Redmond, Colin Wyers wrote that teams might be moving toward inexperienced managers because it’s easier for them to exert control over a skipper who isn’t established. According to Hoyer, though, the trend toward younger managers might have an economic explanation: players make too much money now to be willing to ride buses in the bush leagues after they retire. If teams want former players to manage, they’ll have to be willing to put up with some amount of on-the-job training.

What’s the role of stats in the clubhouse?
According to McCarthy and Javier Lopez, stats can’t be force-fed: if a player isn’t prepared for them, they can cloud his approach. Pitchers who aren’t overpowering may be more willing to embrace analytics because their margins for error are smaller and they’re more open to exploiting any edge, but it’s still easy to reach a point of information overload.

Lopez said something interesting: he doesn’t think that teams are really even trying to persuade players to alter their approaches with stats, or find players who’d be receptive to statistical persuasion. Instead, they’re just targeting players who already do things that the stats say are good. Eventually, he believes that players will realize what approaches teams prefer in their players, and they’ll try to do those things of their own accord, without any nudging from numbers guys.

On the GM panel, Rich Hahn emphasized how easy it is to lose a player’s trust when citing statistics. Even if a statistical approach is sound 99 times out of 100, if it happens to backfire for a player the one time he tries it, he’ll be much less inclined to listen the next time. So when you cite a stat, you have to be sure.

One other tidbit that I’ll add here because it doesn’t fit anywhere else in the article: McCarthy mentioned that he hates facing hitters who confuse him by doing things he doesn’t expect. Among the most confusing hitters to face: Jeff Francoeur, because he swings when no one else would.

Ben Lindbergh is an author of Baseball Prospectus. Follow @benlindbergh
Click here to see Ben's other articles. You can contact Ben by clicking here