April 22, 2013

Baseball Therapy

What is a Good Pitching Coach Worth?

by Russell A. Carleton

the archives are now free. All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible. Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month Recurring subscription - cancel anytime. Subscribe for $39.95 for one year a 33% savings over the monthly price! Purchase a $39.95 gift subscription a 33% savings over the monthly price! Already a subscriber? Click here and use the blue login bar to log in.

Leo Mazzone is a genius. Dave Duncan can take a scrap heap veteran and turn him into something useful. Don Cooper is dreamy. Right?

Let's talk about how we might measure, or at least attempt to measure, the contributions of the major-league pitching coach, the guy who walks out to the mound when the pitcher is on the edge of a meltdown and has a quick conversation that fixes everything. In theory, it's his job to make the pitchers with whom he works better. But how can we tell whether he's doing a good job?

It's unfair to base all judgments of a pitching coach on the outcomes that occur while he is sitting in the dugout. He might be working miracles with a ragtag bunch of guys who belong in Triple-A—limited talent will get you only so far. His team might succeed despite the fact that his philosophy is that pitchers should throw with their non-dominant arm. Steve Carlton could have caught so many hitters off-guard if he had come into the game and thrown right-handed! How to separate the accomplishments of the teacher from the talents of the students?

Fortunately, this is one of those questions that transcends baseball. It's also an important question in understanding the education of children. There is a great deal of debate out there over whether we currently have a good system in place for judging which students are doing well (i.e., the debate over standardized tests as measurements). There’s also a debate over how much credit or blame a teacher should get compared to that given to parents or the community more broadly. But what we do have is a good deal of methodology and math used to answer the education question that we can apply to baseball.

A few years ago, I used this methodology to look at what effect managers might have on their pitchers (admittedly, in a clumsy manner). I've also pulled this out when looking at what effect veteran catchers might have on pitchers. Most of the reason that I looked at managers and catchers was that I didn't have a good list of pitching coaches available. Recently, the kind folks at Retrosheet began listing the names of the coaches that teams have employed over the years at the bottom of each team page. I hope they don't mind that I did a little cut-and-pasting. (Note: I am completely serious when I say this. David Smith and the guys from Retrosheet belong in the Baseball Hall of Fame. The campaign starts here.)

Warning! Gory Mathematical Details Ahead!
I took all pitcher seasons from 1993 to 2012 (again, Retrosheet) and found those pitchers who had faced at least 250 hitters within a season, excluding when the opposing hitter was the pitcher. For those pitchers, I calculated the rates of the three "true" pitching outcomes: walks, strikeouts, and home runs (per batter faced) within that season.

I matched each pitcher with his pitching coach, although with a few exceptions for some amount of mathematical control. Pitchers who changed teams mid-season were assumed to have no pitching coach for that year. The reason is that it allowed their stats for that season to remain in the data set. (For the initiated, I'm about to use an AR(1) covariance matrix, and the pitching coaches are going to be fixed effects anyway, so we'll just ignore that bin. For the people who wandered into the methods section by accident, just trust me. It's a good idea.) I treated pitchers who played on teams who changed pitching coaches mid-season similarly. I also excluded pitchers who played their entire (usually short) career under the tutelage of only one pitching coach.

I only looked at pitching coaches who had 10 or more pitcher-seasons that they had shepherded after all of the above filters had been applied. If a pitching coach didn't have enough students, that pitcher-season was assigned to the "blank" category.

Because we're working across 20 years, and the game has changed somewhat over that time (e.g., strikeouts are now much more common than they were 20 years ago), it was necessary to add a park and a league adjustment to all rates. I took the raw rates for each stat, and weighting by the number of plate appearances the pitcher had at home and on the road, I assigned him an "expected" rate of (road PA percentage * league average for that stat in that year) + (home PA percentage * what visiting teams did when at bat in his park). Yes, I know that the latter term will be conflated with the quality of his teammates. I took the ratio of what each pitcher really did to what was expected of him. (Smith had 105 percent of the league average in strikeouts.) I multiplied that ratio by the league rates in 2012. In this way, we have a somewhat sloppy, but quick and easy league and park adjustment to apply, with everyone being pro-rated to a "neutral park" in 2012.

(deep breath)

I used a mixed linear modeling approach. To control for player quality, I used an AR(1) covariance matrix (pegged to calendar year). I've used this type of control before. It basically sets up a covariance matrix where the model can see that a pitcher who had a high strikeout rate the year before is likely to have a high one the next year. To control for aging curves, I took the player's Opening Day age (April 1st) and entered it as a fixed effect categorical variable. In general, this will allow for the general aging curve of pitchers, even though it is not as clearly defined as that of hitters.

Finally I entered the pitching coach's identity as a fixed effect. I saved the parameter estimates for each pitching coach (80 made the cut) and examined them relative to one another. The output below can be interpreted as "Given an identical pitcher, we might expect that pitcher would be X percent better/worse under this pitching coach, compared to the average pitching coach in our sample."

If you made it this far and have no idea what I just said, I did #math.

The Results

Best Pitching Coaches For Strikeouts	Increase in Percentage Rates from Average Pitching Coach	Worst Pitching Coaches For Strikeouts	Decrease in Percentage Rates from Average Pitching Coach
Oscar Acosta	2.52%	Milt May	2.20%
Phil Regan	2.48%	Mark Riggins	1.98%
Leo Mazzone	1.71%	Ray Searage	1.78%
Greg Pavlick	1.55%	Bill Campbell	1.38%
Rafael Chaves	1.50%	Tim Belcher	1.37%

 

Best Pitching Coaches For Walks	Decrease in Percentage Rates from Average Pitching Coach	Worst Pitching Coaches For Walks	Increase in Percentage Rates from Average Pitching Coach
Johnny Podres	1.10%	Dave Stewart	0.81%
Greg Pavlick	1.07%	Milt May	0.70%
Tim Belcher	0.74%	Gil Patterson	0.69%
Spin Williams	0.66%	Mike Flanagan	0.59%
Don Rowe	0.56%	Oscar Acosta	0.54%

 

Best Pitching Coaches For Home Runs	Decrease in Percentage Rates from Average Pitching Coach	Worst Pitching Coaches For Home Runs	Increase in Percentage Rates from Average Pitching Coach
Charles Nagy	0.56%	Bill Campbell	0.59%
Dave Righetti	0.32%	Mark Riggins	0.42%
Leo Mazzone	0.32%	Milt May	0.36%
Rafael Chaves	0.32%	Rick Adair	0.33%
Ray Searage	0.27%	Fergie Jenkins	0.27%

Well, the model doesn't like Milt May, although May's primary pitching coach experience was in Colorado before the humidor. May was also a former catcher who had primarily served as a hitting coach prior to his stint as a pitching coach. As always, as much as I try to control for the effects of park, there's only so much that the model can absorb before it starts blaming the pitching coach.

But Leo Mazzone and Rafael Chaves (currently the Dodgers’ minor league pitching coordinator) both make two appearances on the "best of" lists (the strikeout and home run lists, specifically). The model doesn't like that Pirates pitching coach Ray Searage has a staff that's lower on strikeouts than expected, but it does like that they give up fewer home runs than might be expected.

For those who would like to see these numbers converted into FIP:

Best Pitching Coach Effects on FIP	Expected Team FIP Benefit over Average Pitching Coach	Worst Pitching Coach on FIP	Expected Team FIP Penalty below Average Pitching Coach
Greg Pavlick	0.38	Milt May	0.45
Rafael Chaves	0.34	Bill Campbell	0.37
Leo Mazzone	0.33	Mark Riggins	0.35
Johnny Podres	0.27	Mike Flanagan	0.20
Phil Regan	0.25	Gil Patterson	0.20
Charles Nagy	0.24	Marcel Lachemann	0.20
Jim Colborn	0.21	Mike Butcher	0.19
Bobby Cuellar	0.20	Dave Stewart	0.17
Dave ​Righetti	0.17	Jim Hickey	0.17
Curt Young	0.15	Frank Funk	0.16

Let's talk about sample size for a minute. As I mentioned, a pitching coach needed only 10 qualifying pitcher-seasons to get on this list. As with anything, the more data points available, the more robust the estimate. For a moment, let's consider Charles Nagy, pitching coach of the Arizona Diamondbacks for the last two years. The model likes Nagy because Diamondback pitchers have seemingly performed better than they should. (Greg Pavlick and Rafael Chaves were both MLB pitching coaches for a couple of seasons each, as well.) But let's remember that if Nagy had been around as long as Leo Mazzone was, we'd be a lot more confident that his performance reflected his true talent level at coaxing performance out of players.

Let's go back to Mazzone. Mazzone's pitchers, on the whole, performed about a third of a run better (as measured by FIP) than we might have expected them to. If we assume that replacement level might be defined as the 31st place on this list (Bob McClure, most recently of the Red Sox, at 0.05 above average), then Mazzone adds .28 runs of value per nine innings above a replacement level pitching coach. If we just take that .28 and multiply by 162 games, we see that Mazzone, according to the model and some quick assumptions, was worth about 45 runs above replacement to the Braves (and Orioles while he was there) as a pitching coach.

Leo Mazzone was worth 4.5 wins per year during his career? That's not superstar level, but there isn't a team out there who would turn away a 4.5 win player who walked into their clubhouse. In 2005, it was reported that when Mazzone took the Orioles' pitching coach job, his salary doubled to $500,000. Even if we assume that the model is overstating Mazzone's contributions (and yes, I am correcting for the fact that he had Maddux, Glavine, and Smoltz) and that he was worth two wins per season, that's still a bargain.

The model also has nice things to say about current Oakland A's pitching coach Curt Young and current San Francisco Giants pitching coach Dave Righetti. Both men have been around in the pitching coaching business at the MLB level for a decade or so, and using the above calculations, have both been worth about two wins a year to their respective teams.

Caveat Pitcher
Let's talk about a couple of things that might throw a wrench into these findings. First, there's more to a team's pitching than its major-league pitching coach. He's important, but he's also backed up by the pitching instructors in the minor leagues. It's also possible that the pitchers are learning from each other or from the manager. Or from one of those Tom Emanski videos.

Another possible confound is that the model will reward pitching coaches who have pitchers that do well when with them, but fall apart with other pitching coaches (sometimes on other teams). It may be that Leo Mazzone happened to be the beneficiary of a Braves front office that was good at predicting when other teams' pitchers would break out (and acquiring them at that point) and/or when their own pitchers would break down (and arranging for them to find other teams). However, it's hard to believe that at least in the latter case, Mazzone might not have been involved in tipping off the front office to a pitcher's imminent decline.

Finally, this model looks only at what pitchers do within the season that they interact with the pitching coach. The model cannot speak to how good a pitching coach is at developing young pitchers over time. It is also an aggregate model. There are probably certain pitching coaches who work well with young pitchers, or sinker/slider guys, or guys with multi-syllable last names. The model sees only the aggregate performance, rather than the fine-grained details of individual cases. The ratings of long-tenured pitching coaches in this system will also be based almost exclusively on free agent and trade acquisitions, because the model requires that a pitcher have experience with more than one coach. For example, Matt Cain and Tim Lincecum (both career-long Giants) do not register on Dave Righetti's radar in this model.

But as always, let's take a look at the forest. A really good pitching coach can have an effect on the order of a couple of wins.  And for someone who works relatively cheap, that's a nice little bargain.

Russell A. Carleton is an author of Baseball Prospectus. Follow @pizzacutter4
Click here to see Russell's other articles. You can contact Russell by clicking here