October 15, 2015

Pitching Backward

Some Signatures Are Forgeries

by Jeff Long

Printer-friendly

Contact Author

The first Giant to get his World Series ring this April was arguably the one who had done the least to earn it. Before the Giants' game against the Diamondbacks on April 18th, Chris Heston opened the black, glossy box, adorned with a gold imprint showcasing the Golden Gate Bridge. Inside was rich black leather interrupted by the crisp lines of the Tiffany & Co. logo and a field of diamonds so large it's hard to believe it's real. Bright diamonds on the ring curveed and shifted directions, creating the SF mark, the same one that adorned Heston’s cap. On the side, below Heston’s name, bright yellow gold and smooth white gold outline San Francisco’s well-known monument; the opposite side showcased a representation of the Giants’ three World Series trophies. "It looked real to me," he said later. He shut the box and went to work.

Heston got his ring first because he was starting that day's game against Arizona, not because anybody would remember his contribution to that championship team. He threw only 5 1/3 innings for the 2014 Giants, split between two relief appearances in blowouts and an unimpressive start in a throwaway Game 162. But on this day, Heston would earn the satisfaction he must've felt when he returned to his locker post-game to finally put the ring on his finger: He threw seven-plus excellent innings, good for a game score of 70. Andrew Baggarly of the San Jose Mercury News perhaps said it best:

Here's one thing you never expected to say on the night the Giants received their World Series rings:

Where in the world would they be without Chris Heston?

It's an interesting question in retrospect. Heston's season was, in a lot of ways, like a World Series ring: There was sparkle and shine, and in the right light it was awe-inspiring--hard to believe, at times, that it was real. On the other hand: It was, at some fundamental level of its existence, not much more useful than a paperweight.

The shine: On June 9^th, Heston threw the season’s first no-hitter:

Game score of 98.

About a month before the no-hitter, he threw a two-hitter against the Houston Astros, striking out 10 and walking nobody. It was a dominating performance.

Game score of 89.

About a month after the no-hitter, he threw a one-hitter against the San Diego Padres over 7 1/3 innings.

Game score of 80.

Three gems. Yet, for the season, his ERA was worse than the league average and he produced just 0.7 WARP in nearly 200 innings. This occasional brilliance but persistent mediocrity piqued the interest of Guy over on Tom Tango’s site:

"Heston—who I also don’t know much about— raises an interesting issue. He had three games of 80 or more this season (using the original formula), equal to Hamels, Greinke, Price, Felix, Lester, Kazmir, Miller, and MadBum. But his season stats are not particularly impressive (95 ERA+, 4.09 FIP). I’ve often wondered whether, for a young pitcher, the ability to throw high-GS games is at all predictive of future performance? Obviously, it matters less than his seasonal stats. But it might indicate a higher “ceiling” than would be true for a pitcher with similar overall stats, but who hasn’t produced any dominating starts."

This echoes an idea that Bill James once raised in his Historical Abstracts. James suggested that certain performances might have "signature significance," a sort of proof of concept even if the product itself is still in Beta mode. As Rany Jazayerli would write in 2009, while testing the signature significance of a dazzling Luke Hochevar performance, "It is the “signature” of a great player, and only a great player, to have such a performance." Of course, the words "Luke" and "Hochevar" probably give away the outcome of Jazayerli's inquiry. Similarly, in 2007, Joe Sheehan came up empty when testing 10 Ks/0 BBs as a signature performance—though he faintly left open the possibility that 13 Ks and no walks might make for significance. "Having one big day didn’t mean anything for these guys," Sheehan wrote. "It was one day. Maybe it was really cold, or maybe they were facing a day-game-after-night-game lineup. Maybe they just applied their particular talents in a way that they never could again. What’s clear, though, is that one big start didn’t herald great things."

With the help of the BP research team, I looked at every pitcher who tossed multiple games with game scores over 75 in a season while delivering a season-long effort that fell below the league average. Perhaps, these moments of brilliance were simply precursors for solid and, importantly, sustained performances in subsequent seasons?

The short answer is no, but first some background on the data. The parameters were that a pitcher must have thrown multiple 75+ game score starts in a season, with that season being among the first three of their careers. The pitchers in question also had to have an ERA worse than the league average, something that eliminates pitchers like Jose Fernandez who are simply very good from day one.

The result is 570 pitcher-seasons since 1953. That includes the Orioles’ Miguel Gonzalez who was almost a run worse than average in 2015 despite throwing a 78 and a 76. It also includes, on the other end of the spectrum, Denny McLain, who threw nine (!) 75+ game score outings in 1966 despite having an ERA nearly half a run higher than the league average. (A note here, for good housekeeping but also to explain McLain a litle bit: Game Score isn't park or era adjusted, so a sub-average pitcher is more likely to throw one, or nine, in a park like Tiger Stadium, in a year like 1966, and in an era where starters work deep into games and start every fourth day.)

The first thing I was curious about was how strong of a correlation the number of 75+ game score outings would have to future performances. I calculated the correlation between the number of very good games and DRA- and WARP over the next five years, 10 years, and balance of the player’s career. The results, in terms of the correlations are below:

The correlations suggest that there might some relationship here, but the question is what exactly does this group look like in terms of performance after these seasons. The average pitcher was worth 5.6 wins in the first five seasons after making this list, roughly 1.1 wins per year. There is, of course, huge variance here. Armando Galarraga and Max Scherzer made the list within a year of one another—2007 and 2008 respectively—but Galarraga was worth -2.1 WARP in the following five seasons while Scherzer was worth 21.1 WARP. In this five-year window the standard deviation is actually larger than the mean (6.7 wins for the standard deviation), suggesting that there are a lot of mediocre pitchers being boosted by extreme outliers like Scherzer. By DRA-, these 570 pitchers were 7 percent worse than the average MLB pitcher over the next five years, reinforcing the idea that most of these pitchers were not in fact special.

The returns over the subsequent 10 year time frame are equally grim. WARP/season drops below one, with this group producing 8.8 wins while being about 8 percent worse than average per DRA-. (This excludes pitchers making the list from 2006 on, because they won’t have had the full 10 years to accumulate value.)

The career numbers don’t look all that different, either. Looking at full careers, the pitchers who met the criteria for the list produced just over 12 WARP on average, with a DRA- just barely worse than league average at 103. The career numbers, of course, give us the most complete glimpse into the guys who really did figure it all out after experiencing a rocky season or two early on in their career. (They're also where survivorship bias is strongest.) Take this group of pitchers who all had a season that met the above criteria:

• Roger Clemens, 1984
• Nolan Ryan, 1968
• Gaylord Perry, 1962
• Don Sutton, 1967
• Mike Mussina, 1993
• John Smoltz, 1990 & 1991
• Steve Carlton, 1968
• Tom Glavine, 1989
• Dave Stieb, 1979
• Jim Palmer, 1966
• Luis Tiant, 1965
• Sandy Koufax, 1958
• Orel Hershiser, 1986
• Cole Hamels, 2009

That list includes some impressive names, and it’s not even all of the eye-catching ones that meet the criteria. The reality, though, is that the majority of the names on the list are less awe-inspiring. Tony Cloninger did it three times from 1962-1964 but would finish with 9.1 career WARP. Russ Ortiz generated 14.1 WARP over his career after earning a spot on the list in 2000. Wade Blasingame's 1965 ERA was only .2 runs worse than league average, but his career generated an ugly -5.5 WARP. All in all, nearly 40 percent of the pitchers who accomplished a Heston finished with fewer than 5.0 career WARP, and one in 10 finished below replacement level. Nearly one in five threw fewer than 500 career innings.

Chris Heston could very well be one of those bullet points above when his career is done; he could also be somebody like Ortiz, which would qualify as a big success. Of course, I’d be hard-pressed to make the argument that he’s more likely than guys like Taijuan Walker or Yordano Ventura, who also met the criteria in 2015. The reality is that a few elite game scores while generally pitching below average isn’t really predictive of anything. Some guys figure it all out, but most don’t.

Those few excellent games sure are fun to look at, just like those impressive names in the list above. The reality is that it belies a much larger body of inconsequence. Chris Heston’s diamond-studded championship ring sure is pretty, but chances are it won’t serve much of a purpose other than to show his grandkids one day. And chances are Heston won't develop into much more than a fun piece of trivia.

In case you're wondering: We also broke this down by offensive and pitching era, and for four-man rotation eras, and so on. The "significance" of the "signature" went up slightly in later eras, which isn't surprising—collecting multiple 75+ game scores got harder. But not by much, and not enough to change out understanding of the achievement. Thanks to Rob McQuown for his research and data assistance. .

Jeff Long is an author of Baseball Prospectus. Follow @JeffLongBP
Click here to see Jeff's other articles. You can contact Jeff by clicking here