June 24, 2004

Prospectus Today

Biased

by Joe Sheehan

One of the reasons we started Baseball Prospectus was to point out the biases within the baseball industry that were affecting player evaluation. We've worked hard to establish the ideas that great athletes don't necessarily make great baseball players, that command is as important to pitching as throwing hard is, and that hitters tend to follow a predictable career path.

We traded infallibility for a package of draft picks, though, so along the way damaging biases have crept into our analyses, the same way that they did in traditional evaluation. If performance analysis is going to continue to make inroads as both a perspective for covering baseball and a decision-making tool for management, its practitioners will have to understand these biases and how they corrupt the process.

The best-known aspect of the modern school of baseball analysis is its emphasis on how often a hitter reaches base. A shift from the old way of evaluating hitters just based on their batting average, or how often they got a hit, the new mindset values walks and the high OBPs they help to create. Where BP has gotten into trouble is in getting overly excited about players with exceptional walk rates who bring little else to the table. Over the years, we've touted players such as Jackie Rexrode and Mark Johnson who had big seasons, featuring high walk totals, in the minors. None of these players were able to replicate that performance at higher levels, with their batting and slugging averages dropping as pitchers learned to challenge them.

Kevin Youkilis, called up last month by the Red Sox, is just the latest example. Youkilis, referred to in Michael Lewis' Moneyball as "The Greek God of Walks," garnered attention for his high batting averages and doubles power, as well as his amazing eye at the plate. Promoted to Triple-A last year, Youkilis was overmatched, batting .165 and slugging .248, and he opened 2004 putting up similar numbers for Pawtucket. He eventually adjusted and was hitting .258 with a .406 slugging average when he was promoted to the majors. With the Sox, Youkilis is at .273 with a .434 slugging. For the first time in his career, Youkilis is striking out more often than he walks. His eye is a great asset, but he'll have to keep hitting at least .270 and drive the ball for doubles to be a viable contributor.

These players make it clear that a high walk rate isn't enough for success. A prospect with a great batting eye has to be able to hit the ball hard enough, often enough, to both have value beyond the walks and to avoid seeing a steady diet of strikes as he ascends the ladder.

At the major-league level, you can see this misplaced emphasis on plate discipline in the A's signing of Scott Hatteberg to a three-year contract. Hatteberg, to whom Lewis devoted an entire chapter, personified the patient approach that the A's want from their hitters. However, he doesn't do enough else at the the plate to be valuable. Despite his plate discipline, Hatteberg was one of the worst regular first basemen in the American League last year. If he hits .319 with power--as he is at this moment--he can be an asset, but that's the point: he has to do more than walk to be a productive player.

Our tools for evaluating offensive performance are well-developed. Our tools for measuring defense are not nearly so precise. Because we have a better idea of what a player contributes at the plate than in the field, we have tended to overvalue offense at the expense of defense. Over the years, BP has advocated that teams should find spots for players who could hit and do virtually nothing else.

Defense matters, though. It's possible to be so bad with the glove, so awkward on your feet, that you're unable to play a position on the field without hurting your team. Jack Cust is a heck of a hitter, but a disaster with the glove at first base and in the outfield. Jeremy Giambi has a career OBP of .377, yet he is so bad defensively that he can't hold a job.

This blind spot is already being fixed. As our defensive metrics improve, we get better at evaluating a player's contributions with the glove. We're getting better at balancing offense and defense in our work, and finding the line where a good hitter gives back too much with his glove to warrant a lineup slot. Moreover, teams on the cutting edge of performance analysis are moving beyond the work of outsiders and developing proprietary systems for gauging defensive value. The A's, who moved from a slugging team to a pitching-and-defense team over three seasons--ditching the same Jeremy Giambi along the way--are at the top of the list of teams working to apply advanced techniques to defense. Their belief in their system motivated their trade for Mark Kotsay and their pursuit of Mike Cameron over the winter.

In the 1980s, Bill James investigated the career paths of baseball players and discovered that hitters had a bell-curve shaped path with a peak around the age of 27. This notion is one of his most important discoveries, and drives much of what performance analysts do, particularly in prospect projection. Performing well while young relative to the other players in the league is a hallmark of a top prospect.

It's also occasionally the hallmark of a guy who just peaked before he was old enough to buy a drink. Javier Valentin hit .324 wth 19 home runs in 383 at-bats in the Midwest League as a 20-year-old. Baseball Prospectus 1997 dubbed him the best catching prospect in baseball. Nine years later, Valentin has 616 major-league at-bats and a career slugging average of .344. Wilson Betemit was our #5 prospect in 2002, coming off his .355 batting average as a 20-year-old in the Southern League. Two seasons later, he has a 48/14 strikeout-to-walk ratio in Triple-A, and growing questions about whether he'll ever hit in the majors.

With the recent spate of players caught falsifying their ages, it makes sense to be skeptical of players who put up great seasons in their teens. They might have great talent, or they might just have their cousin's birth certificate. In the above cases, though, there's no three-birthdays-in-one explanation; the players just didn't develop, and they serve as a reminder that prospect projection remains inexact. Individuals occasionally surprise and disappoint us, which is great. If they didn't, this would be a very boring gig.

It's not just hitters we can get wrong. The emphasis on performance and control of the strike zone that leads to infatuations with certain hitters also leads to the touting of pitching prospects with great strikeout-to-walk ratios. In some cases, though, those pitchers succeed because their superior command allows them to baffle inexperienced hitters. Painting the corners with 88-mph fastballs can work in the majors, but with rare exception, a pitcher has to be able to miss bats to succeed in The Show.

The scouting bias against players of this type is real, and so is the fact that BP has, perhaps in reaction to that bias, been too enthusiastic about people like John Stephens, Ed Yarnall and Mike Meyers. They all put up amazing control numbers in the minor leagues, and all saw their careers peter out well before reaching stardom. The Orioles produced Stephens, Josh Towers and John Maine (currently at Triple-A) in a three-year span, and none can be expected to have more than a token major-league career.

Steve Woodard has a career major-league strikeout-to-walk ratio of better than three-to-one, and better than five-to-one at Triple-A over the past three years. He's found himself in the Red Sox and A's organizations the past two seasons, two teams who use performance analysis to make decisions, and yet he still can't get a major-league job. Minimizing walks is great, but it takes more than command to pitch well in the major leagues. Woodard's "stuff"--that scouting touchstone--isn't good enough to keep major-league hitters from pounding him into submission.

All things considered, performance analysis brings a lot to the table, and a successful team has to incorporate it into its player evaluation. It's not perfect, however, and any application of the method has to include an understanding of the biases it can introduce. Just as an affection for young men who are strong and fast can lead to a system full of decathletes that can't hit a baseball, evaluating players based on walk rates and power can lead to having too many prospects who can't catch one. The best approach, the one that's going to put the most wins on the board, takes the best information from each method.

Joe Sheehan is an author of Baseball Prospectus. 
Click here to see Joe's other articles. You can contact Joe by clicking here