May 14, 2003

Lies, Damned Lies

Randomness: Catch the Fever!

by Nate Silver

Printer-friendly

Contact Author

Your favorite player hit .360 last season. If you know nothing else, what can you expect him to hit this season?

This isn't meant to be a trick question; let's assume the guy had at least 500 at bats in the previous season. Gates Brown and Shane Spencer need not apply. What's your best guess? .350? .340? Not likely.

The evidence is overwhelming. Let's look at all hitters since WWII who hit .350 or better in at least 500 at bats; the only other requirement is that they had at least 250 at bats in the year following.

Name			Year	BA (n) BA (n+1) Delta
-----------------------------------------------------
Aaron, Hank		1959	.355	.292	-.063
Boggs, Wade		1983	.361	.325	-.036
Boggs, Wade		1985	.368	.357	-.011
Boggs, Wade		1986	.357	.363	+.006
Boggs, Wade		1987	.363	.366	+.003
Boggs, Wade		1988	.366	.330	-.036
Boudreau, Lou		1948	.355	.284	-.071
Carew, Rod		1974	.364	.359	-.005
Carew, Rod		1975	.359	.331	-.028
Carew, Rod		1977	.388	.333	-.055
Cash, Norm		1961	.361	.243	-.118
Clemente, Roberto	1961	.351	.312	-.039
Clemente, Roberto	1967	.357	.291	-.066
Cooper, Cecil		1980	.352	.320	-.032
Erstad, Darin		2000	.355	.258	-.097
Garciaparra, Nomar	1999	.357	.372	+.015
Garr, Ralph		1974	.353	.278	-.075
Gwynn, Tony		1984	.351	.317	-.035
Gwynn, Tony		1987	.370	.313	-.057
Gwynn, Tony		1995	.368	.353	-.016
Gwynn, Tony		1997	.372	.321	-.051
Helton, Todd		2000	.372	.336	-.037
Kuenn, Harvey		1959	.353	.308	-.045
Madlock, Bill		1975	.354	.339	-.016
Mantle, Mickey		1956	.353	.365	+.012
Martinez, Edgar		1995	.356	.327	-.030
Mattingly, Don		1986	.352	.327	-.025
McGee, Willie		1985	.353	.256	-.097
Musial, Stan		1946	.365	.312	-.054
Musial, Stan		1948	.376	.338	-.038
Musial, Stan		1951	.355	.336	-.019
Musial, Stan		1957	.351	.337	-.014
Olerud, John		1993	.363	.297	-.066
Olerud, John		1998	.354	.298	-.056
Piazza, Mike		1997	.362	.328	-.034
Puckett, Kirby		1988	.356	.339	-.018
Rodriguez, Alex		1996	.358	.300	-.058
Torre, Joe		1971	.363	.289	-.074
Vernon, Mickey		1946	.353	.265	-.088
Walker, Harry		1947	.363	.292	-.070
Walker, Larry		1997	.366	.363	-.003
Williams, Ted		1948	.369	.343	-.027
-----------------------------------------------------
AVERAGE				.360	.319	-.041
MEDIAN				.358	.326	-.040

There were 42 hitters that met the criteria; on the average, they hit about .360 in the year in question. The next year?

• 38 of the 42 hitters (90%) saw a decline in their batting average;
• 13 of 42 (31%) failed to hit .300;
• The average drop-off in batting average was 41 points; the median drop-off in batting average was 40 points.

We don't need to whip this horse too much, since the data pretty much speaks for itself. It's a commonly stated stathead credo that batting average is subject to a higher degree of random fluctuation than most other components of offensive performance, and numbers like this are the reason why.

The .350+ hitters are certainly an exceptional group, with a few absurdities like Ralph Garr and Darin Erstad (ducking flying halos) tossed in, but they're hardly alone in exhibiting a very profound tendency to regress to the mean.

Using the same criteria as described above, there were 44 hitters who hit between .340 and .350 in a single season. Of those hitters, 39 (89%) saw their batting average decline in the next year; as a group, they hit just over .312.

There were 85 hitters who hit between .330 and .340. Of those, 72 (85%) saw their batting average decline; as a group, they hit .308 the next time around.

There were 144 hitters who hit between .320 and .330. Of those, 118 (82%) saw their batting average decline. They hit, on the average, .301 the next season. And so on.

Here, let's try one of those visualization exercises.

The chart was derived by splitting all hitters into groups of 10 points of batting average apiece, as we were doing above, and comparing the group's average BA from one year to the next. We know about the tendency for very high batting averages to decline, but it's not until a hitter is down to .275 or so that we don't need to worry much about regression. Conversely, if a hitter is much below .260, he's a good bet to improve. While there are some selection bias problems in play here--not every .220 hitter is going to be invited back for another go--the phenomenon certainly cuts both ways.

The next question--and you can probably guess the answer--is whether other offensive statistics exhibit the same pattern. Using the same grouping approach, here's the isobar chart for walk rate.

Walk rate is more stable from year-to-year than batting average. It regresses to the mean, but not as much, which is evident from the flatter bars.

Here's isolated power, which is somewhere in between.

The chart is actually somewhat asymmetrical--a hitter coming off an extremely good power season can expect to drop-off more profoundly than a hitter coming off an extremely poor power can expect to rebound. Still, isolated power is substantially more stable than batting average; for our group of hitters, the year-to-year correlations were .78 and .49, respectively.

I suspect that, in a refreshing turn of events, a lot of the good BP readers out there are going to think this is a rather superficial analysis, but really that's the point. Regression to the mean isn't some lofty concept or novel idea--it's the cold hard fact. If a hitter hit .340 or better last year, odds are more than 90% that he'll experience a drop-off in the following season. That's somewhere alongside a Color Me Badd video on the opposite side of subtle, and it needs to be accounted for when evaluating players.

Randomness--catch the fever! God may not roll dice, but I'm pretty sure he's in one of Joe Sheehan's Strat leagues (he's the one who took a flier on Runelvys Hernandez).

I'm not on a word count, but...

• I was going to write some clever, PECOTA-friendly article about Mike Piazza's potential position shift this week, but there just isn't the data to work with. There are a few famous catchers who switched positions late in their careers--Johnny Bench's name is always mentioned, and you've also got Yogi Berra and Ted Simmons (only Berra experienced much sustained success after his shift). But there aren't very many, and the reason is really very simple--in most cases, it just ain't worth it. Mike Piazza will be the best hitter on the Mets no matter what position he plays. That almost never happens; nobody was yearning to extend Andy Allanson's career.

  I'm not saying there isn't a defensible, quantitative way to answer the question, the chewy part of which isn't what effect the move would have on the Mets (Vance Wilson < Tony Clark) but what effect it would have on Piazza. I am saying that I haven't been able to come up with one.

  But really, it's more of a medical question. Catching is hazardous to a player's health. If you quit smoking at age 34, how much of the damage is reversible?

Nate Silver is an author of Baseball Prospectus. 
Click here to see Nate's other articles. You can contact Nate by clicking here