March 27, 2006

Future Shock

How Do Teams Draft?

by Kevin Goldstein

Printer-friendly

Contact Author

"They're Moneyballers."

"They just take high school players."

"Of course they're good, look at how much money they spent."

These are the kind of things one hears when talking to people about how teams draft. Can we actually measure how teams draft? I gave it a shot.

To do so, I used the bonus data available for the last three drafts, and then totaled the amount of money spent by each team for their top five picks, as defined solely by bonus. So Andy Laroche (a 39th-round selection) is still a top-five pick for the Dodgers in 2003 because of his $1 million bonus. Here are the 10 highest payouts to the top five bonuses in a single draft since 2003:

YEAR TEAM         $(M)  BRIEF EXPLANATION
2005 Diamondbacks $8.57 Justin Upton ($6.1M)
2004 Angels       $6.44 Jered Weaver plus two late-round big money signs
2004 Twins        $5.75 Five picks in first 39
2003 Indians      $5.63 Three picks in first 31 cost $4.56M
2005 Marlins      $5.58 Five picks in first 44
2005 Royals       $5.56 Alex Gordon ($4M)
2004 Diamondbacks $5.55 Stephen Drew ($4M)
2003 Brewers      $5.50 Rickie Weeks ($3.6M)
2003 Devil Rays   $5.48 Delmon Young ($3.7M)
2004 Devil Rays   $5.42 Jeff Niemann, $600K to 10th round

Worth the money? We shall see. Here are the bottom five:

YEAR TEAM         $(M)  BRIEF EXPLANATION
2005 Giants       $0.50 No picks until 4th; 6th and 8th don't sign
2004 Orioles      $1.12 Wade Townsend mess, no 2nd rounder
2003 Phillies     $1.14 No 1st or 2nd; 6th didn't sign
2004 Giants       $1.43 Don't pick until No. 70
2004 Braves       $1.47 No first round pick

The average team spent $3.75 million on their top five bonuses, so for the three-year period studied, you're looking at $11.24 million per team for 15 players, and a grand total of $337 million dollars spent on 450 players who have yet to step to the plate or throw a pitch in a professional game. Here are your top and bottom five spenders for the last three drafts.

TEAM          $(M)
Diamondbacks $17.88
Royals       $14.85
Indians      $14.26
Devil Rays   $14.00
Tigers       $13.95
. . .
Braves        $8.58
Orioles       $8.42
Astros        $7.40
Giants        $5.74
Phillies      $5.58

Interesting note: Not only did the Diamondbacks spend more than three times the amount on their top five bonuses over the last three years than the Phillies and Giants, they spent more money on one player (Justin Upton), than either of those teams did on their 15 players.

So knowing which teams spend a lot and spend a little is easy to figure out, but it's also not the best measurement of how teams draft. Rather, it's the best measurement of where teams draft. For the most part, the teams spending the most money were those consistently picking near the top of the draft, where bonuses get into the multi-million range. In other words, consistently bad teams. In order to come up with a second measurement on how teams draft, I tried to measure patterns of risk. To do so, I enlisted the assistance of Rany Jazayerli, who has done a valuable series here at BP measuring draft picks. Based on Rany's research, I asked him to assign a "risk coefficient" for each kind of selection, assuming the safest pick is a college hitter, and here's what he came up with:

 College Hitter:  1.000
 College Pitcher: 1.501
 Prep Pitcher:    1.506
 Prep Hitter:     1.621

Using these values, I assigned each player a value that I called RiskCost. I used these risk coefficients in the same way that degree of difficulty is used in competitive diving. So by multiplying the bonus by the risk coefficient, I came up with RiskCost values for each selection. Here are the 10 highest RiskCost scores for a single player over the past three years:

YEAR TM  PLAYER          BONUS   DOD  RiskCost
2005 ARZ Justin Upton     6.10  1.621   9.89
2004 LAA Jered Weaver     4.00  1.501   6.00
2003 TBD Delmon Young     3.70  1.621   6.00
2005 NYM Mike Pelfrey     3.55  1.501   5.33
2004 SDP Matt Bush        3.15  1.621   5.11
2003 DET Kyle Sleeth      3.35  1.501   5.03
2004 TBD Jeff Niemann     3.20  1.501   4.80
2004 DET Justin Verlander 3.12  1.501   4.68
2004 NYM Philip Humber    3.00  1.501   4.50
2005 DET Cameron Maybin   2.65  1.621   4.30

Justin Upton's RiskCost score is the highest by over 50% and also the highest of all time. What you can see here is that with risk brought into the equation, the system implies that Cameron Maybin at $2.65 million is a riskier selection than the $4 million bonus that Stephen Drew and Alex Gordon, both college hitters, received.

Using this data I assigned a weighted Risk Score to each team. Instead of simply adding up the 15 RiskCost values for each team, I added them and then divided that total by the total amount of money spent. In other words, the risk of signing a high school hitter for $2 million has four times the impact on a team's risk score than signing a high school hitter for $500K.

So now we have two measurements for each team: a total in money spent, and a weighted risk score. This is the kind of data that screams for a scatter plot:

So now we can view how each team deals with the draft on both money and risk levels, with cash outlay on the Y axis and weighted risk on the X axis. By splitting the chart into four quadrants using the averages of all 30 teams, I've categorized each team into one of four types. Let's look at each one.

Big, But Cautious Spending

Best Example On A High Spending Level: Arizona Diamondbacks

The Diamondbacks have spent more than $3 million more than any other team with their 15 bonuses. This is mostly because of the record bonus ($6.1 million) handed out to 2005 No. 1 overall pick Justin Upton, and the $4 million given to 2004 first-round selection Stephen Drew. But you have to hand it to Arizona and scouting director Mike Rizzo, because they haven't been afraid to do what it takes to get the best talent. By taking Upton last year and Drew at No. 15 overall in 2004 (when other teams were afraid to deal with agent Scott Boras), the Diamondbacks exercised patience and guts and wound up with the equivalent of two No. 1 overall picks. Despite Upton being the 'riskiest' pick in the study (and arguably the one with the highest upside) Rizzo has been exceedingly safe otherwise. Upton is the only Arizona draftee among the top 15 bonuses to come out of the high school talent pool, and the three players with the highest bonuses after Upton--Drew, Carlos Quentin and Conor Jackson--are all college hitters.

Best Example On A Low Risk Level: Oakland Athletics

Is it any surprise to BP readers that the team with the lowest weighted risk is Oakland? Seven of the top eight bonuses paid by the Athletics over the last three years went to college hitters. Keep in mind, that even Oakland may be coming around to the fact that this kind of drafting is possibly too risk-averse, as the only reason their plot isn't even further to the left is that they spent over $1.5 million on a trio of high school arms in 2005.

Other Teams:

• Brewers: They spent a ton of money because they've picked in the top five in each of the last three drafts. Taking college hitters (Rickie Weeks and Ryan Braun) have kept their risk down.
• Indians: While the Indians have handed out six million-plus bonuses over the last three years, their top four bonuses have been to given to college players, with three of those being position players.
• Rangers: Texas's top bonus over the last three years is $2.1 million to high school lefty John Danks, but the team rarely spends big on high school hitters, with outfielder K.C. Herren ($675K in 2004) being their biggest buy.
• Red Sox: The largest bonus handed out by the Red Sox over the last three years actually went to a high school player (Mike Rozier), a 12th-round pick in 2004 whose pro debut last season was a complete disappointment. Their four other seven-figure bonuses went to college players, three of them hitters.
• Rockies: I was surprised to see the Rockies to the left of the average risk line, as when I think of their drafts, I think of first-round high school picks Ian Stewart (2003) and Chris Nelson (2004). However, their top bonus ($2.3 million) went to college shortstop Troy Tulowitzki, and the majority of their 2nd-5th round picks have gone the college route.
• Royals: 2005 first-round pick Alex Gordon and his $4 million bonus is the only thing keeping the Royals in the low-risk category, as their next four highest bonuses all went to high school talent.

High Rollers

Best Example On A High Spending Level: Tampa Bay Devil Rays

Because the team always stinks at the big league level, the Devil Rays always pick high, and man, do they let it ride. Of the 15 bonuses looked at here, not a single one was handed out to a college hitter, and only three went to college pitchers. More than half of their $14 million spent have gone to two players: the best prospect in baseball (Delmon Young), and a high-ceiling pitcher (Jeff Niemann) who has had problems staying healthy so far.

Best Example On A High Risk Level: See Devil Rays, Tampa.

Other Teams:

• Angels: Like the Devil Rays, the Angels tend to avoid college hitters like the plague, with 2003 fifth-round pick Blake Balcom ($170K) the only one among the top five bonuses for his year. They also spend heavy on pitching. Five of their top six bonuses have gone to arms, with uber-prospect Brandon Wood the lone exception.
• Marlins: The Marlins had five picks in the first 44 last year, and used every one of them on a pitcher, four of them from high schools. Like the Devil Rays, not one of their top five bonuses in any one year went to a college hitter.
• Mets: The Mets' top five bonuses paid have gone to four college pitchers (including a total of $6.55 million for Philip Humber and Mike Pelfrey) and Lastings Milledge.
• Tigers: Unlike the other teams here, the Tigers are not against taking college hitters--they just wait until the second round. Picking second, third and 10th in the last three years has turned into over $9 million for Kyle Sleeth (Tommy John surgery in 2005), Justin Verlander (best righty prospect in baseball?) and Cameron Maybin, who had far too much promise to pass up when he fell to Detroit.
• Twins: The Twins are seen as kind of the anti-A's by some, but while their top two bonuses went to high school hitters (Matt Moses and Trevor Plouffe), three of their next four went to college pitchers. The Twins definitely avoid college hitters, and while they ranked 11th with $12.7 million spent, that is a function of the Twins losing free agents and having handfuls of extra picks late in the first round and in the supplemental stages. They've paid eight bonuses of $875K or higher, but nothing over $1.5 million.

Spend Little, Risk Little

Best Example On A Low Spending Level: San Francisco Giants

The Giants have spent less than $6 million on their 15 bonuses, and they've deliberately timed moves in the last two years in order to sacrifice their late first-round picks. This is a practice that borders on criminal when you look at the kinds of players the Giants could have taken with this pick. The next time there is a meeting among the Giants executives, and somebody starts talking about the worthlessness of a late-first round pick, can somebody stand up and remind that person it was the 25th pick in 2002 that netted the team Matt Cain? Please? Anyone? Bueller?

Best Example On A Low Risk Level: Washington Nationals

Like the Royals, the Nationals have one player/bonus (Ryan Zimmerman/$2.975M) skewing their risk score, but they do avoid high school players pretty consistently. 2004 3rd-round pick Ian Desmond ($430K) is the highest bonus they've given to a high school hitter, while Collin Balester is the only prep pitcher they've given a six figure bonus to.

Other Teams:

• Blue Jays: The king of the so-called "Moneyball" teams, all 15 of Toronto's top-five bonuses have been handed out to college players, but their weighted risk is a little higher than one might expect because four of their top five bonuses have gone to college pitching.
• Mariners: The Mariners' risk is skewed by 2005 first-round pick Jeff Clement, a college hitter who got $3.4 million. The Marlins have had few picks at the top, and high school hitter Matt Tuiasosopo ($2.29M) is their only other seven-figure bonus and their next four highest bonuses are all prep talent.
• White Sox: The reigning champions have used their top picks (all getting bonuses from $1.55-$1.6 million) on college talent, two of them (Brian Anderson and Josh Fields) hitters. One of the rare times they've delved into the high school market, they found highly regarded prospects (Gio Gonzalez and Ryan Sweeney).
• Phillies: The Phillies have paid only one bonus greater than $1 million in the last three years, handing out $1.475 million to toolsy outfielder Greg Golson in 2004, their only first-round pick in the three-year period studied. They've also had only one second-round pick, and their three highest bonuses after Golson have all gone to college hitters.

Big Risks, Little Investment

Best Example On A Low Cost Level: Houston Astros

Tulane lefthander Brian Bogusevic is the team's only first-round pick in the last three years, and as a general rule, the teams strays away from high school hitters with their early picks. What brings their risk score a little above average are a pair of high school pitchers taken in the late rounds who received second round money. 2003 13th-round pick Jimmy Barthmaier ($750K) and 2004 9th-round pick Troy Patton ($550K) show that the Astros can be aggressive in taking prep players with signability issues.

Best Example On A High Risk Level: New York Yankees

The Yankees always pick late, but they definitely look for high-risk/high-ceiling guys when they do invest, with their top five bonuses all going to prep players, and four of those going to high school hitters. 2005 third-round pick Brett Gardner is the only college hitter the Yankees have selected above the ninth round in the last three years, and big 2005 bonuses to late picks Austin Jackson and Alan Horne may be a sign of things to come as the Yankees look to parlay their high-spending ways into the draft without the benefit of ever picking towards the top.

• Braves: Here's an interesting one. With their only true first-round pick in the last three seasons, the Braves took college closer Joey Devine, who is also the only college pitcher the club has taken in the first four rounds since 2003. The next seven bonuses after Devine all went to high school players, five of them pitchers.
• Cardinals: St. Louis is really all over the board, with no discernable trend, really. They really seem to focus on just taking the player they like best--an admirable trait--giving out million-plus bonuses to a college pitcher (Chris Lambert), college hitter (Tyler Greene) and prep hitter (Colby Rasmus) while using a supplemental first and two second-round picks to nab high school arms.
• Cubs: The Cubs' costs are kept down by a lack of a first-round pick in 2004, and no second-round selection in 2003. They haven't selected a college hitter in the first five rounds of either of the past two drafts, and nearly half of their money went to a pair of highly-regarded high school players, Ryan Harvey ($2.4M in 2003) and Mark Pawelek ($1.75M in 2005).
• Dodgers: The Dodgers are only with this group because of Luke Hochevar. If Hochever had actually signed the deal ($2.98M bonus) he agreed upon while temporarily under the advisement of agent Matt Sosnick, the Dodgers would be firmly in the High Roller category. The Dodgers have given out seven-figure bonuses to three high school arms (Chad Billingsley, Scott Elbert and Chuck Tiffany), while never spending more than $325K on a college position player.
• Orioles: Baltimore's total cost is low because of the 2004 Wade Townsend fiasco. The risk is high as over 40% of their top 15 bonuses went to two players--high school catcher Brandon Snyder and teenage junior college sensation Nick Markakis.
• Padres: Matt Bush skews the San Diego risk score because of his $3.15M bonus. Other than 2004, when the Padres used their first three picks on prep talent, the team has been almost exclusively college-oriented, waiting until the 26th round to draft a high school player in 2003, and the 15th in 2005.
• Pirates: Drafting 11th or higher in each of the last three drafts, the Pirates have pretty much spent the league average on bonuses, but the big money has gone to a college pitcher (Paul Maholm), and a pair of high school hitters (Neil Walker and Andrew McCutchen). While Maholm's bonus ($2.2M) is the highest the team has paid in the last three years, it may be an outlier. He's the only college pitcher the team selected in the first four rounds.
• Reds: Like the Cardinals, the Reds are all over the board, with their top four bonuses since 2003 going to one of each of the four basic types of players. They really have established no pattern here, but the lack of stability in the front office could be the main cause of that. If 2005 is any indication, they avoided college position players in the future -- they didn't take one until the seventh round.

One thing that was not addressed here is a judgment on these four groupings. Is any one of them a right way to draft? If a team is spending a lot, does it make sense to spread your risk around? If for whatever reason a team has a limited budget for the draft, does it make sense to go for the riskier players who may have a higher ceiling? If you look at the chart again, you can find very good and very bad systems in every quadrant, except one. The Spend Little, Risk Little teams have spent less than most and put the money into low-risk players--putting little impact talent into their organization, and leaving all of those teams lagging behind when it comes to overall system strength.

So there is more than one way to draft properly, and much of it depends on draft position, available talent in any one year/at any one draft position, and obviously the collective skills of a team's scouting director and their staff. From evaluating drafts and prospects for years, my gut tells me that--all things being equal--the ideal draft balance on this chart is a curve that begins at the top somewhere round around the Diamondbacks' point, slopes to where the average risk and cost intersect, and finishes on the right below the Yankees a tick or two.

But it's only March, and there is plenty of time for more studies.

Kevin Goldstein is an author of Baseball Prospectus. 
Click here to see Kevin's other articles. You can contact Kevin by clicking here