August 31, 2010

Expanded Horizons

Understanding Post-season Odds

by Tommy Bennett

As the calendar changes over to September, baseball fans focus on the eight playoff spots and the race to secure them. For a certain type of shoot-from-the-hip, never-tell-me-the-odds kind of fan, this is a freewheeling season of derring-do and thrills. For a different type of fan—a type of fan I daresay is more likely to visit this website—the uncertainty of September practically cries out for odds making and handicapping.

Fortunately, we at BP have some pretty nifty ways to do just that, using a mix of simple and complicated methods. Let’s start with a discussion of the odds that a particular team will make the playoffs. Each morning, a little after 7 Eastern, the hive mind buried deep inside our Secret Volcano Fortress rumbles awake and spits out our Postseason Odds Report. It’s one of my favorite BP features, if only because it is so intuitive and easy to understand. Of course, the simplicity of its outputs—nominally just percentages—belies the complexity what happens behind the scenes.

Systems are best understood by their oddities, and our PS Odds is no exception. Take, for example, the Cardinals. On Monday morning, they stood three games out of the National League wild-card hunt, behind two other teams (the Giants and the Phillies). Our odds gave the Cardinals a 13.5 percent chance of nabbing the wild-card spot. That is to say, approximately once every six or seven tries, the system is predicting that the Cardinals would overcome both teams ahead of them and win the wild card. Not bad, right?

Compare that with the system’s expectations for the Phillies, who were just two games behind the NL East-leading Braves on Monday. Despite the narrower gap separating the Phillies and their target, as well as the lack of an intervening team, the Phillies enjoy an only marginally higher likelihood of winning the NL East than the Cardinals do of winning the wild card. The odds report predicted them to win the division just 19.4 percent of the time, a relatively modest improvement over the Cardinals’ odds. For those of you who are merely time traveling, sentient androids from the future, maybe the explanation for the seemingly tiny difference is patent (then again, maybe you should watch out because if there’s one thing I learned from robot movies, it’s that human beings are not to be trifled with). But for the rest of us, I think, the similar result despite the rather big difference in the standings is surprising.

I said we could learn a lot from a system by looking at its outliers, but that’s not really right. Those data points we choose to select as outliers are a reflection of our prior understanding, which may or may not be a good baseline from which to compare. So let’s look at the internal workings of the system to understand why things have come out this way.

There are three things that are absolutely critical to know about the odds report. First, they rely on third-order winning percentage. Second, they rely on a Monte Carlo simulation. Third, they are only as smart as their inputs. Let’s take those three facts one at a time.

A Third Order—Of Awesome

Third-order winning percentages are calculated in our Adjusted Standings Report and are a variant of a Pythagorean record. In fact, the differences between a simple Pythagorean record—which uses only runs scored and runs allowed as its inputs—and a third-order winning percentage, are really just twofold. First, instead of looking at how many runs a team actually scored, third-order winning percentage looks at how many runs we would have expected the team to score based on their TAv for and against. Various artifacts of sequencing and good fortune, like hitting particularly well with runners in scoring position, are thus washed away. But secondly, and perhaps even more importantly, third-order wins then adjust the underlying True Average numbers to account for the quality of opponents faced. For example, if a team (we’ll call them the Blue Jays) mustered a .262 TAv while facing some of the best pitchers in the league, then they get a little boost in the third-order department.

So that’s why it’s third-order, because we adjust for both expected runs (second-order) and quality of opponents. Think of it as the Steven Lukes of winning percentages (I’ll be here all week; don’t forget to tip your sociology professors). There is one other twist: Instead of slavishly using the same expected winning percentage each time, the BP hive mind actually regresses the third-order winning percentage back to the mean—to account for the familiar tendency of hot teams to cool off and vice versa. This creates a good projection of future performance, which we can think of as an expected winning percentage.

Can Somebody Please Teach Me How to Play Baccarat Chemin-De-Fer?

Our playoff odds also use what is known as a Monte Carlo simulation, which is a relatively simple computational algorithm that is often used to predict future outcomes (they can run pretty straightforwardly in Excel).  The essence of the Monte Carlo method—as the gambling provenance of its name hints—is random sampling. It’s the same method used by many financial ratings agencies, investment banks, and hedge funds to evaluate risk in their portfolio, but don’t let that fact sour you on it.

What the Monte Carlo method allows for are changes in teams’ expected winning percentages. Each time the odds report runs one of its one million simulations, it randomly selects a winning percentage from a normal distribution around the expected winning percentage. In plain English, the computer randomly sets an expected winning percentage for each team that is close to, but probably not identical to, the expected winning percentage discussed above. That means each of those one million simulations is unique (or rather, the odds of any two being identical are very, very low), and it’s also why it’s useful that the computer runs the simulation so many times—to smooth away the effects of one odd sample.

The odds report also knows each team’s schedule from here to the end of the season, and uses what is known as the log5 method to calculate the odds one team (with a randomly sampled winning percentage) will beat another team. It’s a handy little approximation that does the work quickly. One neat feature of this is that the Playoff Odds takes into account the future strength of a team’s schedule, as well as the strength of its schedule in the past.

Know What You Don’t Know

As with all systems, it’s critical to know what the system doesn’t know. It doesn’t know about roster changes—whether they be minor league call-ups, trades, or waiver claims. Really, all it does is what I’ve described above. It also doesn’t know about individual pitching matchups, which can be important if managers are good at setting their rotation to maximize their chances of winning.

Question of the Day

All this should help explain what is going on with the Cardinals and the Phillies. The Cardinals are chasing the Giants and the Phillies, who both have third-order winning percentages that are lower than their actual winning percentages (a difference of four wins for the Phillies, and five wins for the Giants). Meanwhile, the Cardinals have a third-order winning percentage right in line with their actual winning percentage. Moreover, they have an easier remaining schedule than either the Giants or Phillies, who play in more competitive divisions.

On the other hand, the Phillies are trying to catch the Braves, who have a very strong third-order winning percentage (.568). However, the Playoff Odds Report does not know about the loss of Chipper Jones and his .297 TAv for the year. All of these factors combine help explain why the Phillies are only moderately more favored to overtake the Braves than the Cardinals are to leapfrog two teams to win the wild card.

My hope is that you can now explain the Postseason Odds Report to your mother (she is just upstairs after all, right?). Are there any other surprises with this system and its methodology?