College World Series

Evaluating College (and Major League) Pitchers

I want to expand on one of the themes I touched on last week and show how a metric made necessary by the very large canvas of the college game can actually show us something about the professional game as well. In the process, I'll introduce a new metric to the BP audience. I realize this is quite a bit like the guy hired as the caterer for the Food Network set pushing this really neat recipe he has for Scrambled Eggs a la Bert, but bear with me; there's a useful point in here somewhere.

What's the best statistical way to tell which starting pitcher has performed the best? Not who's the best pitcher, or who has the most potential, or who would be more likely to succeed if we replayed the season, but who's actually been most valuable to his team? Most people would go with ERA, perhaps with some adjustment for innings pitched. Some adjustment might be made for home park, if we want to get a bit more complicated. The more sophisticated in the crowd might go with something like SNWL, VORP, or ARP, but it's interesting how closely rankings based on those measures tend to track with rankings based on ERA.

When you start trying to use ERA to compare college pitchers, though, even with park factor adjustments, you'll quickly discover that a given ERA can mean vastly different things depending on the context in which it was produced. Facing down Enormous State U. and Prospect Tech is quite a different game from going against Vine Covered U. and Our Lady of the Perpetual Small Endowment.

As an example, let's look at the 2001 season, since we've got a little distance now to begin to verify our original impression of the players involved. The NCAA ERA leader for 2001 was a kid named Todd Pennington from Southeast Missouri State, with a 1.33. Down in fourth was Southern California's Mark Prior at 1.69. There were several other good pitchers in the top 10, although Dewon Brazelton and Prior are the only ones to make their major league debuts thus far. Now, Todd Pennington's not a bad pitcher at all; he's a command-and-control junkballer who's put up a 1.06 ERA as a closer in the Sally League so far this year. Prior, though, was being called the greatest college pitcher of all time by people with enough historical knowledge that there was a chance it was true.

In order to find a better way to get a better statistical feel for whether the subjective ordering was likely to be more accurate than the ERA-based one, I decided to take a look at a very simple metric--Runs below Opponent Average. RBOA is not at all complicated; for each start, simply take the runs that the opponent would score in an average game for them during that season in the number of innings that the pitcher throws and compare it to the actual runs allowed. For example, if a pitcher allows two runs in six innings against a team that usually scores six a game, he picks up two RBOA. I use runs against for two reasons--the notion of earned and unearned runs causes at least as many problems as it solves, due to the vagaries of official scoring (especially in college), and finding average earned runs scored for each team would be a pain.

Looking at RBOA, Prior did indeed put up an absolutely phenomenal year, including the following stretch:


Opponent         RPG     IP     R

Washington       6.0     9.0    1
Arizona          7.9     9.0    0
San Diego State  6.3     5.0    2
Arizona State    8.3     8.0    1
California       6.3     9.0    0
Stanford         6.9     9.0    1
UCLA             6.6     9.0    0

All told, his RBOA total for the year was 70.7. If you adjust for 2001 college scoring levels and assume that 12 runs is roughly equivalent to a win, Prior was worth around six wins over a 70-game season, an astonishing total. Meanwhile, Pennington's ERA, while impressive, was achieved against the likes of Tennessee Tech, Eastern Illinois, and Tennessee-Martin; his RBOA total was only 32.8.

Now, how does this lesson apply to the big leagues? I mean, major league pitchers all face equal competition over the course of even a partial season, right? Well, as it turns out, no, they don't. Let's take a look at the AL for this year so far. Here are the ERA leaders as of Tuesday morning, with their RBOA included:


Pitcher     ERA        RBOA

Loaiza      1.96       20.96
Mulder      2.36       20.48
Mussina     2.62       22.42
Hernandez   2.79        8.30
Martinez    2.83       11.83
Sabathia    2.92       13.23
Zito        2.94       15.19
Cornejo     3.00        9.04
Lohse       3.09       11.57
Meche       3.11        9.44

Esteban Loaiza has faced the following teams: Detroit three times, Kansas City, Baltimore twice, Minnesota twice, Seattle, Oakland, and Toronto. Meanwhile, Mike Mussina has faced these teams: Toronto (the AL's leading run-producing offense) three times, Minnesota twice, Texas twice, Seattle twice, Anaheim, and Boston.

As a partial-season metric, RBOA seems to have some value, then. What about over the full course of the season, though? Will those inequities in opposition strength even out? As it turns out, probably not. A quick look at the White Sox schedule to date shows that, while they've managed to avoid the Yankees and Red Sox so far, they also haven't faced Tampa Bay yet. More relevantly, the percentage of games they've played against their divisional opponents is roughly proportionate to the percentage for the whole season; with the unbalanced schedule, difficulty of opposition really never does even out.

One other side issue worth noting, because it might lead to an interesting conclusion, is Barry Zito's position on the RBOA list. He's ahead of the guys around him on the ERA list not just because of oppositional quality but because, at the time the list was constructed, he was the only AL pitcher to have made it to 12 starts. If we assume that the rough shape of the quick-look data above holds and a start by a top-line pitcher is worth about two runs above a start by an average starter, then getting your best starter the ball five more times is worth around a win. That might be an interesting data point to look at when investigating the question of a four-man vs. five-man rotation.

On the college news front, we're into the super-regional level this weekend, where the 16 regional winners pair off for best-of-three knockout series for the right to advance to the College World Series in Omaha. Due to underseeding of a couple of Western teams, there are a couple of great matchups this weekend, as Arizona State travels to Cal State Fullerton and Long Beach State heads up to Stanford. ESPN is broadcasting the North Carolina State-Miami game at noon Eastern on Saturday and regionally-available games on Sunday and Monday afternoons, along with the rest of the weekend's action on Pay per View. Keep an eye out.

Boyd Nation is the sole author and Webmaster of Boyd's World, a Web site devoted to college baseball rankings, analysis, and opinions. In real life, he's an information security analyst with an energy company. He's writing a series of articles for BP on the college game and the College World Series. He can be reached at boyd.nation@mindspring.com