November 11, 2014
Grading the 'MLB Executive' Predictions, 2012 and 2013
Two years ago, Jerry Crasnick released his annual Polling of the Execs, in which a couple dozen GMs and GM-types ("general managers, assistant GMs, player personnel people and scouts") answered that ol' impossible request: Predict Baseball. It's one of my favorite columns of the offseason, giving us insight into the way that people on the inside think baseball works. I used the occasion to look back at those GM-types' success rate over the previous nine seasons, and found that they were hardly better than random chance at predicting whether a player would be good ("I saw (Kaz) Matsui four years ago, and I thought he was a better player than Ichiro,") or where a player was going to be signed or whether a player was going to be traded or to whom. Don't mistake that for lack of value; just acknowledge that what we're getting is a particular type of insight from these answers, if not necessary a particularly bankable forecast.
There's a new batch out today—as fun as always: Lester > Scherzer, predicted A-Rod home run totals, etc.--so let's take the occasion to go over the past two years of new data we have: The 2012 offseason predictions, and the 2013s. Unless it becomes clear that the GM-types have turned a corner and require full reassessment, we'll keep the methodology light and just assign a snap judgement, right or wrong.
Q. Which team will sign Josh Hamilton? For how much money and how many years?
They said: The Brewers were the most common answer, "from a low of three years, $60 million to a high of five years, $130 million.
Grade: D. Of 22 respondents, none said the Angels. The deal he signed is just barely within the range of answers, though we don't really know the distribution of those answers. For all we know 21 GM-types all said "five years, $130 million."
Q. Which team will sign Zack Greinke? For how many years and how much money?
They said: Half said the Angels, "a five- or six-year deal, for a total payout in the $100 million to $150 million range."
Grade: B-. The Dodgers were the second most common answer. Greinke's deal was, again, in the range offered, but (at six years, $147 million) just barely. If we assume the consensus answer is in the middle of Crasnick's range, then the predicted prices were mostly low.
Q. B.J. Upton and Michael Bourn -- which player would you rather sign?
They said: Bourn over Upton, by a roughly two-to-one margin.
Grade: A-. Neither has been good, but this quote about Bourn---"Better defensively. Better OBP. Better makeup. Safer pick"--gets to the correctness of the pick. Oddly, though, Upton got 56 percent more guaranteed money in his deal.
Q. Justin Upton, James Shields and Chase Headley all have been mentioned as potential trade candidates. Which player do you think is most likely to get dealt this winter?
They said: Shields 15; Upton 5; Headley 1.
Q. Do you think Alex Rodriguez will be traded this winter?
They said: All but one said no.
Q. After Zack Greinke, which will be the best signing -- Anibal Sanchez, Edwin Jackson, Kyle Lohse, Ryan Dempster or Dan Haren?
They said: Sanchez 14; Jackson 3; Haren 3; Lohse 1.
Grade: A-. It's not just that they got the right answer, but that they overwhelmingly got the overwhelmingly right answer. That said, Lohse is two tiers above the others but was undervalued in this exercise.
Q. How many years and how much money will Melky Cabrera get?
They said: Most executives expect Cabrera to sign a one-year, make-good deal for a base salary of $2 million on the low side to $7 million to $10 million on the high side.
Grade: B. More or less got the post-PED discount right, but missed on the length.
Overall, a very solid year. A 3.3 GPA.
Q. Where will Robinson Cano sign, and what kind of deal?
They said: The Yankees (more than 90 percent of respondents), "seven or eight years with an overall payout between $160 million and $230 million."
Grade: D-. Not one person said Seattle would sign him, and he signed for more years and for more money than even the highest estimate here.
Q. Which of Jacoby Ellsbury or Shin-Soo Choo -will provide better value over the course of his next deal?
They said: Choo 12; Ellsbury 8.
Grade: D. It's obviously hard to assess this one so early in each player's contract, and "better value" is squishy enough to debate anyway. But Choo was barely replacement level in his first season, while Ellsbury was well above average. If they had been close then we'd have passed on issuing a judgment, but they weren't close.
Q. Which of Matt Garza, Ervin Santana or Ubaldo Jimenez do you like the most?
They said: Garza 10, Santana 5, Jimenez 4. Two said "they will all be way overpaid."
Grade: B? I don't know. I guess Garza was the best, though Santana threw the most innings, and Jimenez was clearly the worst and received the fewest votes, but did receive some votes; can't even say the "all way overpaid" guys got it right, considering Santana was at most moderately overpaid at one year, $14 million. I don't think anybody really got anything or missed anything here. Just a jumble of meh opinions about meh pitchers.
Q. Phil Hughes or Joba Chamberlain -- which has a better chance of blossoming?
They said: Hughes, 6 to 1.
Q. Which team will sign Masahiro Tanaka? Does he get more than Yu Darvish's $60 million deal with Texas?
They said: Most said the Dodgers, though the Yankees were the second most common guess. Most said he would get more than Darvish.
Grade: B-. Didn't get the right team, but almost did, and he got nearly $100 milion more than Darvish did--though the change in posting fee rules was part of that.
Q. Will the Tampa Bay Rays trade David Price this winter? If yes, what's his most likely destination?
They said: More than 80 percent said the Rays would trade Price.
Grade: D+. Incidentally, not one of the "yes" voters said he would be traded to the Tigers.
Q. Which aging pitcher has the most left in the tank: Roy Halladay, Tim Hudson, or Hiroki Kuroda?
They said: Kuroda, with a modest lead over Hudson. Halladay got one vote.
Grade: A. Kuroda and Hudson had almost the same amount left in the tank. Kuroda had slightly more left in the tank. Roy Halladay had nothing left in the tank. This is a good distribution of predictions.
So all in all, a 2.4 GPA for 2013. Not so great. Put it together, and we have
Five questions with an either/or answer. The GM-types got 1 correct.
Four "where will he sign" questions. The GM-types got none correct.
Four questions with multiple possible answers. The GM-types arguably got all four correct, by varying margins.
No conclusions. We'll make our conclusions after 30 or 40 more years, so long as Crasnick keeps blessing us with these.
Sam Miller is an author of Baseball Prospectus.
Click here to see Sam's other articles.
You can contact Sam by clicking here