Moonshot: Prospect Trajectory as Data

In 2007, Reid Brignac was a sure thing. Lacking only elite defensive skills, hounds such as BP’s own Kevin Goldstein praised Brignac’s bat with the most effusive verbiage. With an average ranking of 25th on a collection of top prospect lists, the only question regarding Brignac was at which position he’d be posting an All-Star batting line.

As any recent student of baseball will know, this was not to last. The next year, his ranking had fallen to 32^nd, then 89^th the following year. Brignac’s offense collapsed somehow, and never recovered. A quick perusal of Brignac’s player comments reads like the arc of a cannonball: soaring upward toward stardom, lingering for a moment as a high-end, blue-chip prospect, and then suddenly crashing downward into irrelevance. After having amassed an unimpressive -0.8 WARP in lifetime value, Brignac was last seen signing a minor-league contract with Marlins.

One could contrast Brignac’s parabolic path with fellow Tampa Bay uber-prospect Evan Longoria’s meteoric rise. After going from college draftee to High-A to Double-A in the space of less than a year, Longoria managed to rise into Triple-A in his second year, and was ready for the majors shortly thereafter. Longoria spent so little time in the minors that he merited only two BP comments before he was Rookie of the Year.

A more fair comparison might be to Andrew McCutchen, another bat out of high school. Instead of a rise and fall, McCutchen’s was a slow and steady ascent toward stardom. His average prospect rankings go like this: 65, 20, 18, 27.

The variation in paths of prospects good and bad brings to mind the broader question of how prospect trajectories reflect the ultimate destiny of players. It’s easy to develop narratives around a player’s growth or stagnation, ascribing inevitable success or failure on that basis. To establish the truth of the matter, we must take a more general, less anecdotal look.

Using data from Chris St. John, we can look at the relationship between prospect trajectory and lifetime WARP. As a reminder, St. John’s huge collection of prospect data covers more than 20 years, aggregating multiple lists per year. From this data, I average the ranks across lists for each year to get an overall picture of the consensus on each prospect.

We also need to operationalize the notion of a prospect trajectory. What does it mean for a prospect to ride an upward trend, or slide downhill? To get a first approximation, I take the averaged prospect rank in a player’s first year on the lists in comparison with their last year. The difference in position can then be compared to the player’s lifetime WARP.

To define the risers from the fallers, I grouped prospects by whether they gained 10 or more ranks, stayed within 10 ranks of their initial position, or lost 10 or more ranks.

The results are neatly linear. Those prospects who fell by 10 or more ranks from their first to their last years offer the worst results, at 4.48 WARP. Those who stayed within 10 ranks of their initial appearance on the lists gather an intermediate amount of WARP. Finally, those who climb do the best on average, producing a full 15 WARP, not a bad major-league career.

Instead of placing the prospects in discrete categories, we can also examine the full breadth of trajectories. Seeking to establish a relationship between trajectories and WARP, I plot here the difference in rankings against the lifetime WARP of each player, with a best-fit curve (LOESS) in red.

There’s a surprisingly smooth and continuous curve which links prospect trajectory to future performance. Guys who ascend the ladder of prospectdom, who crawl all the way from the bottom to the top, have an expected lifetime WARP of more than 10. Meanwhile, those who manage to chart the reverse course, from the top to the bottom, are doomed to lifetime production around or below replacement level.

It’s worth noting that the curve asymptotes (flattens out) at both ends, albeit for different reasons. At the lower end, we are seeing the effect of survivor bias. Bad players cease to accumulate further negative WARP, because by definition, replacement level players can exceed their performance.

On the other side of the curve, risers on the prospect rankings tend to cap out at 12 or so WARP. That seems to be a product of the fact that those prospects, despite their incredible upward paths, must have started from humble beginnings. Players who eventually arrive at Hall of Fame-type careers almost invariably come from the top of the prospect rankings, and so have no room to rise.

The relationship between prospect rank difference and lifetime WARP is not perfect, and it explains relatively little of the variance (about 10 percent) in lifetime WARP. But 10 percent is a considerable amount, given the overall difficulty in projecting prospects. If we pull this information together with the handful of factors I identified in previous work (things like age, average rankings, position, etc.), we arrive at an updated projection that gets as far as explaining 20 percent of the variation in lifetime production.

The above finding illustrates that trajectory is partially redundant with other factors related to lifetime WARP. Using only static information (age, average ranking, and so on), about 18 percent of the variation can be explained. So, even though trajectory explains 10 percent of the variation alone, its impact is dramatically diminished when combined with other factors (the technical term for this is collinearity). Still, if we limit ourselves to examining trajectories only of prospects with similar final rankings (within the top 20), trajectory remains important[1]:

The above table shows that trajectory is significant, even for prospects who ultimately attain nearly the same rank. Those who rise to attain their rank tend to achieve better outcomes than those who stay the same, and still better than those who drop.

Overall, this information suggests we ought to pay a little more attention to the direction of a prospect’s rank, and not only their single-year outcome. When we see the same names ascending the lists, year upon year, that’s an affirmative signal for their future potential. Conversely, the first hint of decline from a top prospect ought to be troubling.

This advice must apply more severely to prospects coming from high school than from college. As the Brignac/Longoria story above illustrates, college bats tend to come to the minors with more polish, and spend fewer years inhabiting the rankings. Those coming straight from high school, on the other hand, are likely to spend several years being evaluated, and their upward or downward migration upon the prospect rankings carries more significance. We may never be able to foresee a prospect like Brignac’s decline, but perhaps we can better identify it as it happens by keeping a careful eye on a prospect’s past as well as their present.