March 20, 2008

Lies, Damned Lies

PECOTA Takes on Catching Prospects

by Nate Silver

Printer-friendly

Contact Author

I've kept you all waiting so long for the PECOTA Takes on Prospects series that I'm going to eschew any lengthy philosophical discussions. Instead, let me quickly tick off the new features that should make these rankings more accurate-or at least less inaccurate-than ever before:

1. New Davenport Translations: At least eighty percent of the work in the minor league PECOTAs is really in the Davenport Translations stage. Clay spent extensive time this winter evaluating his translation algorithms, and came to the conclusion that the DTs had not built in a steep enough difficulty curve, particularly for players at the lower end of the minor league food chain. What this means in practical terms is that you're going to be seeing fewer of those oddball forecasts in which some kid who played at Staten Island last season is projected to put up a 750 OPS in the majors next year. The flip side of this, however, is that if players in the lower minors are starting out from a lower baseline, that also gives them more room to develop. In some sense, what these new DTs do are really allow PECOTA to do its job, as it bears more responsibility for figuring out just which types of skill sets tend to experience significant improvement and which ones do not. The high-K-rate, high-walk-rate type of minor league pitcher, for instance-think Clayton Kershaw-is one of those skill sets that tends to improve dramatically, so you should see players like Kershaw expected to struggle in the next year or two but thereafter experiencing significant improvement in their forecast line.
2. Expanded Comparables Database: Another upshot of Clay's hard work this winter is that we have retrofitted the DTs all the way back to 1991. This roughly doubles the size of PECOTA's minor league database, and allows us to be much more precise in selecting comparables. The big question, of course, is which player is going to be the first to get Michael Jordan as a comp.
3. Draft Slot / Signing Bonus Information: This is the one that we've teased a little bit. PECOTA now uses the amount of a player's signing bonus as a factor in determining comparability. We have signing bonus info for the Top 250 picks in the amateur draft going back to the early 1990s, and on a selective basis back to 1987 (where we don't have information on the specific money that a player signed for, we use his draft slot instead as a proxy). This turns out to make a lot of difference: a hitter who was picked at the very top of the draft can get another 10-15 points of EqA appended to his forecast versus a player who is otherwise his equal but was a late-round draft pick. And that's in the n+1 year alone, with the difference tending to grow from there: a player like Justin Upton, for instance, has literally twice the Upside score that he would have had if we didn't account for the draft bonus info. This new feature necessitated a special category for Latin American players, whom it would not be fair to lump into the same group as late-round draft picks. Latin American players tend to be compared to other Latin American players first, whereas US and Canadian amateurs generally won't be compared to Latin Americans. All of this is a tacit admission, by the way, that statistics alone cannot do all the work; the reason why the draft bonus info is so useful is because it serves as a proxy for all sorts of scouting information that PECOTA might not otherwise be able to get a handle on. We'll explore this theme in more detail throughout this series.
4. Seven-Year Forecasts and Six-Year Upside Scores: As you have probably seen from the PECOTA cards, we now explicitly forecast seven years of a player's development rather than five-that carries us all the way forward to 2014. As a result of this improvement, we have now expanded the Upside window from five to six years, which is a more natural fit for the six years of service time that a player may receive before he becomes a free agent. For more information, see our glossary entry on Peak scores.
5. More Regression to the Mean: For players with extremely limited professional performance records-fewer than about 300 lifetime plate appearances as a pro-PECOTA now uses a hybrid model that combines a player's actual statistics with the means of players with similar scouting attributes (height, weight, draft slot, and defensive position). The effect of this adjustment is fairly subtle, frankly, and will mostly be manifested in the absence of small sample-size-players from the elite levels of the PECOTA prospect rankings, particularly those who were not taken with a high draft slot.
6. Forecasts for Recent Draft Picks with No Professional Track Record: In fact, the same technique taken to its logical extreme allows us to create forecasts for players with no professional playing time at all (think Matt Wieters or David Price), based solely on their height, weight, draft slot, and defensive position. While these forecasts are inherently much less robust than "regular" PECOTAs, they generally turn out to be commonsensical. Price, for instance, is compared to other tall college lefties who were taken with one of the top couple of picks in the draft, and his forecast is based on how those players performed over time.

I've been really excited about these improvements, so much so that they led me to spend way too much time working on the PECOTAs, and not enough time writing about them. So, let's get started on the catchers. For ground rules, which are the same as last year, please see last year's introductory article.

Excellent Prospects

Geovany Soto, Cubs (25)       166.9
Jeff Clement, Mariners (24)   149.3
J.R. Towles, Astros (24)      115.0
Matt Wieters, Orioles (22)    105.2

These are, in some order, the group of elite catching prospects in baseball. You can make some arguments for ordering these four players in a different way than PECOTA has ordered them, but you can't really make an argument that anyone outside of this group belonging in the top four.

All the arguments against Geovany Soto deserving a high ranking are fairly specious. Yes, he hadn't done much prior to 2007. But volumes of research have shown that as long as you take your weighted averages properly-and that's the sort of thing PECOTA is good at-having one bad year followed by one fantastic one is no better and no worse than having two average years consecutively. Yes, Soto was repeating his level at Iowa-for the second time, actually-but the DTs build in specific adjustments for repeaters, so that is already accounted for.. Yes, Sosa is a tiny bit old for a prospect, but it's not atypical for catchers to develop slowly. Although he does not make Soto's comparables list, the guy I keep thinking about is Jorge Posada, who has a similar range of skills and did not become a big league regular until the age of 26.

Last year I discussed James Loney, and argued that a player who is advanced up the minor league ladder too quickly may suffer in the near-term, but may see abnormally large gains later on as his skills catch back up with his level. Jeff Clement had barely 200 professional plate appearances under his belt when he was forced up to Triple-A Tacoma in 2006, and would seem to be a pretty good example. PECOTA actually thinks Clement would outhit Kenji Johjima right now, and he probably has the most offensive upside of anybody in this group save perhaps Matt Wieters. The issue is his defense; only 74 of Clement's 125 games in Tacoma last year were at catcher, and PECOTA really dings catchers who get a lot of playing time at non-catcher positions.

I got into an interesting discussion with Cory Schwartz of MLB.com about the relative merits of J.R. Towles and Geovany Soto. Cory rightly pointed out that Towles is a year younger than Soto, and had a considerably better track record heading into 2007. But, there was more of a difference between Towles and Soto last season than you might think. While Towles was outstanding over a couple of months at Corpus Christi (and pretty good in a cup of coffee with the Astros), he also had 115 plate appearances at Salem and another 50 at Round Rock in which he didn't hit at all. That's not really an indictment of Towles, who probably has the most well-rounded skill set of any of these players. But I do think he ranks behind the first two players in this group.

All things considered, I'm fairly happy with what PECOTA did with Matt Wieters, one of the big test cases for how the system was going to handle players with no professional playing time. What's interesting is that Wieters actually has a rather low Beta (0.75), suggesting that his performance is relatively predictable, even though the method we use for dealing with recent draft picks should inherently produce forecasts with big spreads of variance around them. Elite college hitters, almost to a man, tend to be low-risk, medium-upside picks.

Very Good Prospects

Hank Conger, Angels (20)      75.2
J.R. House, Astros (28)       62.2
Jesus Montero, Yankees (18)   60.9
Jose Morales, Twins (25)      51.7

Well, this is an odd little quartet of players. Catcher and shortstop are the couple of positions where PECOTA will throw an older, organizational player into the mix from time to time. Sometimes, this has actually worked out pretty well; we have detected the merits of players like Shane Victorino, and Carlos Ruiz this way. Ruiz, interestingly, is J.R. House's top comparable, although somehow the comparison doesn't feel quite right, as House is significantly behind Ruiz as a receiver. Major league teams seem to have real hang-ups about going with hit-first backups at the catcher position; guys like Robert Fick, Eric Munson, and Matt LeCroy have always had to struggle for opportunities.

Hank Conger is a far more traditional sort of prospect, and has an interesting set of comparables headed by Jarrod Saltalammchia and also including Brian McCann and Paul Konerko. He's a long ways away, but Conger hit plenty well enough in the Midwest League to keep pace with expectations. Jesus Montero is much younger and much more raw than even Conger; his comparables run the gamut from Guillermo Quiroz to Aramis Ramirez, which tells you all you need to know. Jose Morales, like House, is more in the organizational player category.

Good Prospects

Landon Powell, A's (26)           47.4
Taylor Teagarden, Rangers (24)    40.4
John Jaso, Rays (24)              38.8
Bryan Anderson, Cardinals (21)    37.9
Chris Stewart, Rangers (26)       37.8
Francisco Cervelli, Yankees (22)  36.8
Josh Donaldson, Cubs (22)         36.2
Wilson Ramos, Twins (20)          34.9
Jake Fox, Cubs (25)               34.8
Mark Wagner, Red Sox (24)         33.3
Lou Marson, Phillies (22)         32.1
Curtis Thigpen, Blue Jays (25)    30.8
A.J. Ellis, Dodgers (27)          30.0
Wilkin Castillo, D'Backs (24)     29.6
Nick Hundley, Padres (24)         27.9
Lou Palmisano, Brewers (25)       26.5
Matt McBride, Indians (23)        26.1
Jason Jaramillo, Phillies (25)    25.2

You say you can outhit Brad Ausmus and you're still in your twenties? Come on down! The long tail of the catcher position reveals a lot of no-names, but there are nevertheless a few intriguing prospects here. One guy I like particularly is Josh Donaldson, who may have been the most productive prospect in the Pioneer League on a per at-bat basis. Bryan Anderson is a guy that PECOTA liked a whole bunch last year, but he suffered between a year in which he didn't show much development and our new, harsher DTs. John Jaso is the sort of guy we might have gotten more excited about a couple of years ago, with his .408 OBP in the Southern League. His top comparable is Rusty Greer, which is a little weird to envision at the catcher position; the problem is that vision would give you a pretty accurate rendition of how Jaso fields his position. And then there's Landon Powell, who is so old that, like, they didn't have YouTube and stuff when he was in high school, and nobody knew who Zak Efron was. Unlike many of the other players on this list, Powell is a real backstop, and while not reaching Double-A until the age of 25 is usually a strike against a player, he could be an interesting Plan B if Kurt Suzuki flashes out.

Other Notables

J.P. Arencibia, Blue Jays (22)   19.1
George Kottaras, Red Sox (25)    17.3
Mike Rabelo, Marlins (28)        15.4
Max Ramirez, Rangerrs (23)       10.7
Anthony Recker, A's (24)          6.3
Devin Mesoraco, Reds (20)         4.9

Tony Recker and Max Ramirez were the two players from this group who made Kevin's Top 10 list last summer. PECOTA doesn't like Recker because his season completely fell apart after a promotion to Midland, and it doesn't like Ramirez because he's a marginal fielder with much of his value is tied up in "empty walks"-bases on balls that reflect more of an ability to exploit wild pitching in the low minors, and less a reflection of real pitch-recognition skills. Devin Mesoraco also gets a particularly harsh forecast; PECOTA only saw 155 plate appearances from him, but that was enough for it to conclude that he isn't likely to develop a bat.

The Big Picture: Rankings Combined With Non-Rookies 25 Years Old Or Younger

 1. Joe Mauer, Twins (25)                 358.5
 2. Russell Martin, Dodgers (25)          269.9
 3. Brian McCann, Braves (24)             226.0
 4. Geovany Soto, Cubs (25)               166.9
 5. Jeff Clement, Mariners (24)           149.3
 6. Jarrod Saltalamacchia, Rangers (23)   144.0
 7. J.R. Towles, Astros (24)              115.0
 8. Jesus Flores, Nationals (23)          113.4
 9. Matt Wieters, Orioles (22)            105.2
10. Chris Iannetta, Rockies (25)           98.0

You want a deep sleeper in the last couple of rounds of a fantasy draft? Jesus Flores, a Nationals Rule 5 pick a year ago, is an interesting option. While there is not much to look at on the surface in his batting lines, PECOTA reckons that he already has major league-average power, which for a 23-year-old catcher is quite precocious. Chris Iannetta would have had a pretty good year last season if he'd added about 50 points of batting average. Of course, you can say that for nearly anyone, but since Iannetta hadn't had any trouble hitting for average in the past, he's still the best catcher on the Rockies roster (even if his top PECOTA comparable is Ben Petrick). At the top end of the spectrum, meanwhile, Joe Mauer has taken a little hit to his value. PECOTA was once convinced that Mauer would develop the power to pop 15-20 homers, but for each year that that he doesn't, it becomes a progressively less likely outcome for him.

Nate Silver is an author of Baseball Prospectus. 
Click here to see Nate's other articles. You can contact Nate by clicking here