September 24, 2009

The Biz Beat

Fixing the GM Rankings

by Shawn Hoffman

Printer-friendly

Contact Author

Way back in August of 2009, I tried to evaluate major league front offices on how efficiently they were spending their payroll dollars, building on the work of Doug Pappas and Nate Silver. I called it MR3/ExpMR-which was horrible. So from now on, we're going with Payroll Efficiency Rating, or PER. So there.

Anyway, I think we nailed the high end. Teams like the Rays and Red Sox dominated the top of the charts, while the previously overrated Marlins and underrated Yankees converged in the middle. But on the bottom end, there was still a major bugaboo, as I mentioned:

If there's still a piece missing, it's the value that comes with finishing last. The first pick in the draft is worth a lot more than the fifth or the tenth or the fifteenth, so a team that wins 59 games, as the Nationals did last year, should have that factored into its marginal revenue figures.

So there's still some work to do, but the process is actually pretty straightforward: figure out which pick each team should expect to get, based on their third-order win total, and about how much that pick is normally worth, in terms of future value. (For this, I used Sky Andreychuk's draft-pick value calculator, which holds up pretty well under heavy testing.)

Before we get to the numbers, let's acknowledge the five-hundred pound elephant that's about to walk into the room: this adjustment rewards teams for sucking. In fact, the worse a team is, below somewhere around 75-80 wins, the better they'll come out in the final rankings. So maybe we're rewarding incompetence, in some cases. But the reality is that teams are usually better off trying to win 60 games than 81-81 is a moral victory, while 60 could get you Justin Upton.

So with that said, here's the bottom ten with the old system (but updated for current third-order winning percentage), followed by the new one. Also, a big hat tip to Eric Seidman, who ran simulations to find expected draft positions, based on teams' win totals.

Bottom 10, Old Way
Team        PER
Padres     0.90
Nationals  0.90
Cubs       0.89
Brewers    0.88
Orioles    0.84
Royals     0.80
Mets       0.76
Pirates    0.76
Astros     0.75
Reds       0.75

Bottom 10, New Way
Team        PER
Orioles    0.96
Reds       0.96
Indians    0.95
White Sox  0.95
Mariners   0.94
Brewers    0.93
Tigers     0.84
Astros     0.82
Cubs       0.80
Mets       0.72

For the curious, here's the full data. The Pirates (15^th in the new rankings), Padres (17^th), Nationals (19^th), and Royals (20^th) move out of the bottom ten, which shouldn't be a big surprise, given their expected draft positions. Meanwhile, the Mets, Cubs, and Tigers all fall a few places, with the Mets now bringing up the rear. If you're surprised to see the Tigers there, just check out their third-order winning percentage, which would have them winning 78 games. Also-and are you allowed to do two parentheticals in a row? Whatever-you'll notice that the numbers are higher in the new version, but this is actually just a function of the top and bottom converging. The 30-team average is actually down, from about 1.17 to 1.09.)

At the top, the Rays keep the top spot, despite their troubles lately (the third-order standings still have them ahead of the Red Sox), while the Dodgers, Rockies, Red Sox, and Cardinals round out the top five. The top eight teams in terms of third-order winning percentage are all in the top half of the league, and all but the Phillies are ahead of the no-cost Marlins.

So integrating draft-pick values didn't rock the boat at the top of the chart, which I think is a good thing. But there are a lot more questions we can now try to answer with this data. For example, where is that key cutoff point where your team is actually better off losing? We can take the expected pick value curve, add the totals to Nate Silver's marginal revenue curve, and see where the dips are:

Some notes:

• This isn't perfect. It probably overstates the value of winning 75-80 games, for example. But remember something: this doesn't take into account the expense side. Using the 60 versus 81 example from above, the two are very close on the chart. But all other things being equal, an 81-win team should cost you more than a 60-win team. So while the "revenue" side might be similar, the ROI should be much higher for the 60-win team, and it's obviously far less risky. 

• There's a steady drop between 60 and 70 wins, because a team's chances at the number-one pick start going down very quickly in that range. In Eric's simulations, teams that won around 60 games had about a fifty percent shot; teams that won 70 didn't get it a single time, in over 65,000 tries.

• It's still a lot better to win 100 games than 50.

If there's another obvious practical application with this, it's to show how much better it is for a small-market team to win than to simply fold up the tent and go for the number-one pick. Even with draft picks factored in, the Rays are still an easy number one, and the Pirates, Nationals, A's, et al are in the middle of the pack. In terms of pure return on investment, a winning team always justifies a few more payroll dollars.

However, it's also important to remember the constraints that these teams are working under. For example, if the Pirates were on pace to win 92 games instead of 62, we would expect them to bring in about $35 million more in revenue; that's a decent chunk of money, but try improving your team by thirty wins in free agency-where a win usually goes for $3-5 million-with only $35 million. The only way that small-market teams can build economically viable contenders is through player development or trades; we already knew that, of course, but it's an important reminder for Pirates fans who are about ready to hang themselves after watching the team go 3-22 over the past few weeks. (For what it's worth, the Yankees' expected revenue would go up by $140 million if the team went from 62 to 92, which makes their strategy actually seem somewhat reasonable.)

There are plenty of other potential applications of this, which I'm sure I'll get to at some point. In the meantime, if anyone has ideas, fire away in the comments section.

Shawn Hoffman is an author of Baseball Prospectus. 
Click here to see Shawn's other articles. You can contact Shawn by clicking here