September 24, 2013

Baseball ProGUESTus

Was Joe DiMaggio Streaky?

by Jim Albert

Most of our writers didn't enter the world sporting an @baseballprospectus.com address; with a few exceptions, they started out somewhere else. In an effort to up your reading pleasure while tipping our caps to some of the most illuminating work being done elsewhere on the internet, we'll be yielding the stage once a week to the best and brightest baseball writers, researchers, and thinkers from outside of the BP umbrella. If you'd like to nominate a guest contributor (including yourself), please drop us a line.

Jim Albert is a Statistics professor in Ohio. Since he grew up in Philadelphia, he's faithfully followed the Phillies (even this season). He wrote a book, Curve Ball, with Jay Bennett explaining statistical thinking in baseball and has written an introductory statistics text based on baseball. Currently, he is the editor of the Journal of Quantitative Analysis of Sports, which publishes papers describing statistical studies addressing questions in a wide variety of sports.

Joe DiMaggio's 56-game hitting streak in the 1941 season is one of baseball’s most famous events. This record has fascinated many people, including many academics. Much of the discussion of the accomplishment centers on how often a streak of this magnitude should occur, and whether we should expect to see it equaled or surpassed in our lifetimes. But this article is not about the extreme nature of DiMaggio's improbable feat. I'm not going to compute the probability of a batter going through a 56-game hitting streak during a season. Rather, I'm wondering whether DiMaggio's pattern of hitting during the 1941 season, at the at-bat level, was unusually streaky.

Since the distinction between game-level streakiness and at-bat-level streakiness can be confusing, let me explain in terms of coin flipping. Suppose you flip a coin 500 times. I simulated this process on a computer and found some interesting streaky patterns. I had a streaky of eight consecutive heads, six occurrences of six consecutive heads, and so on. Should I be excited by these clusters? Of course not—I'm just observing some streaky patterns that are a by-product of chance. A coin is truly consistent in that the chance of heads is always 50 percent and outcomes of different flips are independent.

Let's move to the hitting outcomes of a baseball player like Joe DiMaggio. We observe his sequence of individual hits and outs during the 1941 season, but unlike the coin, we don't really know DiMaggio's true hitting behavior. There are two possibilities. Maybe DiMaggio is truly consistent (like the coin) and the chance that he gets a hit does not change across the season. Or maybe DiMaggio is truly streaky—this would mean that during part of the season, he’s hot and hitting with a higher-than-average probability, and other times, he’s cold and hitting with a lower probability. By the way, baseball players and managers believe in hot and cold hitting, and managers make decisions about lineups on the basis of these beliefs about hot and cold.

Until recently, we had access only to DiMaggio's game-to-game hitting data for the 1941 season. But this summer Retrosheet made available play-by-play data for some early seasons, including 1941. We now can look more carefully at DiMaggio's sequence of hit/out data for each of his at-bats. We can use this 1941 data to see if DiMaggio's pattern of hitting is any different from that of a truly consistent hitter with a constant chance of hitting throughout the season.

Here are DiMaggio’s outcomes in his 541 official at-bats during the '41 season.

H O H O H O O H H H O H H O H H O O O H O O O O O H H O H O H O H H H H O O

O O O O O O O O O O H O O H O O O O O H O O H O O H O O H O O O O O O O O H

O O O O O O O O O O O O H H O O O H H O H O O H H O O O O O O O H O O O O H

O H H O O H H H O O H O O O H O O O H H O O O O H O O O O H O O O H H O O O

H H H H O O O H O O H O O H O H O O H O O O O O H H O O H O H O O O O H O O

O H O H H O H H O O H O H O O O H O O H O H H O H O O O O H O O O O H O O H

O H H H H H H H O H O O O O O H O H O O O H O H O O O O O H H O H O H O O H

O O H O O O O H O O O H H H O O O O H O O H O O O H H H O H H H O O H O H H

H O H O H H O O H H O H O H O O O H O O H O H H H O H O O O O H H O O O O O

H O H H O O H H O H O O H O H H O O H O O H O H H O O H H O O O O O O O H H

H H O O O O H O H O O O H O H H H O O O H H O O O O O O O O O O H H O O O H

O O O O H O H O O O O O O O O O O O O H H H O O H O O O O O O O O O O H H O

H O O H O H H O O O H O O O O O O O O O H O O O H O O H H O O O O O H O H H

O O H O H H O H O O O O O H O O O O O H H O O H O O H H O O H H O H H O O H

O O O O H O H O O

Baseball fans like to look at "Ofers"—in the 1941 season, DiMaggio had two 0-for-10 slumps and also three 0-for-12 slumps.

Should we be surprised at these mini-slumps in his hitting? It is not clear at this point if DiMaggio is showing evidence of being truly streaky in the season or just exhibiting the same type of streaky patterns that we would see in coin flipping.

Here is one way we can answer this question. First, we think of some way of measuring streakiness or clumpiness in a sequence of hits and outs. There are different ways of doing this, but we'll use a streaky measure equal to the sum of squares of the gaps between hits. In DiMaggio's sequence, the gaps between hits (the Ofers) are equal to:

1 1 2 1 1 3 5 1 1 1 12 2 5 2 2 2 8 12 3 1 2 7 4 1 2 2 3 3 4 4 3 3 3 2 2 1 2 5 2 1 4 3 1 1 2 1 3 2 1 1 4 4 2 1 1 5 1 3 1 5 1 1 2 2 4 3 4 2 3 1 2 1 1 1 2 1 1 3 2 1 1 4 5 1 2 1 2 1 2 2 1 2 7 4 1 3 1 3 10 3 4 1 12 2 10 1 2 1 3 9 3 2 5 1 2 1 1 5 5 2 2 2 1 2 4 1 2

So our streaky measure would be

1^2 + 1^2 + 2^2 + ...+ 2^2 = 1638

I claim that this is a reasonable way of measuring the degree of streakiness in DiMaggio's hitting. To give you some intuition, here are two patterns of hitting.

Player 1: HOOHOOHOOH

Player 2: HHOOOOOOHH

Both players have a batting average of 0.400 = 4/10 over this period. But Player 1 has a consistent pattern of hitting and Player 2 had two clumps of two hits separated by an 0-for-6 slump. The streaky measure is 2^2 + 2^2 + 2^2 = 12 for Player 1 and is 6^2 = 36 for Player 36. This measure seems useful in distinguishing consistent and streaky patterns of hitting.

Next let's suppose that DiMaggio was a truly consistent hitter and his chance of getting a hit stayed the same throughout the 1941 season. If this were the case, then each possible permutation or mixing of the 193 hits and 348 outs would be equally likely.

We perform a simple simulation experiment to determine what patterns of streakiness we'd see from truly consistent hitting. We randomly mix up the order of the hits and outs (the 193 H's and the 348 O's) in DiMaggio's sequence and compute the sum of squares of the gaps between hits. We repeat this simulation procedure, say a thousand times, and we get an idea about the possible values of the streaky statistic if he was a consistent hitter.

I graphed the 1000 values of the streaky statistic in the simulation and displayed DiMaggio's value of 1648 on top.

What do we see? Since DiMaggio's actual streakiness value is in the middle of this distribution, we conclude that the amount of streakiness in his sequence of hits and outs is similar to what we would predict if he was truly a consistent hitter. So there is little evidence to suggest that DiMaggio was truly a streaky hitter during the 1941 season.

Am I saying that DiMaggio’s Great Streak is not remarkable? No. He was a great hitter in the 1941 season, and his greatness led to this game-by-game hitting accomplishment. (In fact, if one does a similar analysis on a game-by-game basis, DiMaggio appears very streaky.) But if one looks at his sequence of hits and outs at an at-bat level, there is little evidence to suggest that his hitting ability changed during the season.

That raises the question: Are there players in the 1941 season who were unusually streaky on an at-bat-to-at-bat basis? The Retrosheet data is not complete for this season, but (using our method) we find that Red Rolfe displayed unusual streakiness, and Tommy Henrich—fittingly enough, nicknamed “Old Reliable”—displayed unusual consistency.

Looking at the 1941 data for Red Rolfe, he had 47 pairs of consecutive hits, and 0-for-17 and 0-for-21 slumps. He appeared streaky since he had an unusual number of clusters of hits and also had long slumps. In contrast, Tommy Henrich had only 34 pairs of consecutive pairs of hits, and his worst slump was an 0-for-10. Henrich had a very consistent pattern of hitting that season. Our method is helpful for picking out streaky hitters and also for finding unusually consistent hitters.

All of the calculations here were performed using the open-source (free) statistical system R. On the website bayes.bgsu.edu/dimaggio, the interested reader can find an R script, dimaggio.R, that can be pasted into the R console window to reproduce these calculations. BP author Max Marchi and I have a new book, Analyzing Baseball Data with R, coming out in November that gives a baseball introduction to R and provides a variety of sabermetric illustrations using the R environment.