June 13, 2013

The Stats Go Marching In

Measuring Catcher Framing in the Minor Leagues

by Max Marchi

In my last article, I presented the results of using Retrosheet pitch-by-pitch data for measuring catchers’ framing performance. After showing that the alternate method fared quite well, despite not relying on pitch location data, I went on to provide historical leaderboards (Brad Ausmus is tops among catchers of the past quarter century) and explore the issue of aging (Father Time seems not to take much of a toll on framers).

I left you with one teaser: while it was nice to have some of the retired catchers ranked, the most valuable byproduct of that research was that it made ranking active catchers at lower levels possible. That’s the topic I’ll tackle today.

Framing in the minors
New PITCHf/x systems are continuously being installed in minor league ballparks, so analysts working in MLB front offices probably have the luxury of rating minor league catchers using at least some pitch location data.

However, despite the relative richness of their databases, before putting a dollar sign on the mitt they’ve likely had to ask themselves the same question I’ll attempt to answer here: Is it safe to use framing numbers recorded at the minor league level to project success (or failure) at mastering the art in the big leagues?

One of the obstacles to measuring minor league framing is the wider variation in the competence of the players on the field: some of the pitchers (likely those who will not keep climbing the ladder) have very poor control, making framing a lost cause for their batterymates. Umpires are also likely to be less consistent than their colleagues at calling balls and strikes in the majors, making framing and its measurement more difficult. Add to that equation the continuous movement of players throughout farm systems, which makes catchers less likely to be familiar with the repertoires of their teammates on the mound, and you have some idea of the challenges facing a statistical approach to measuring minor league receiving skills.

So, is this a lost cause? Let’s see what the numbers say.

I used data from games played at the Triple-A and Double-A levels (since lower tiers have more scattered pitch sequence data) from 2005 to 2012 to compute minor league framing data. Then I matched the numbers of each catcher to the ones he compiled in the majors over the same span, using the same Retrosheet-based method. Here’s what the results look like.

Note that I’ve changed the prorating to 10,000 pitches, which should be a more sensible translation to a regular catcher’s full season, as Tom Tango suggested after my previous article. Note also that, as I have been doing lately, I shaded data points according to their underlying sample size (in this case, the lower number between pitches caught in the minors and pitches caught in the bigs).

Catcher framing numbers in the minors seem to predict those obtained in The Show fairly well: a weighted correlation coefficient computed on the data points shown above, comes out at a healthy 0.58. How does that compare to other statistics?

Taking the same timeframe (2005-2012) and the same sample of players (catchers), we get 0.17 for batting average. The rest of the triple-slash stats are only slightly more reliable: the correlations for OBP and SLG are 0.22 and 0.26, respectively.

Bottom line: if you take unadjusted values, you are going to have an easier time inferring framing skill at the major league level from minor league numbers than you will inferring offensive performance.

Retrospective and prospective
Many of the catchers you have become accustomed to seeing at the top of framing leaderboards used to do the same in the minors. Russell Martin, Ryan Hanigan, David Ross, and Miguel Montero all saved over 20 runs per 10,000 pitches while in the minors.

The three names that come up the most in framing articles are Jose Molina, Jonathan Lucroy, and Ryan Doumit (the last one for very different reasons than the first two). Unfortunately, the middle brother of the great Molina dynasty played his minor league career in years for which we have no pitch sequence information available, so we can’t look at how he did as a prospect.

However, we do have over 10,000 pitches caught in the minors for poster boy no. 2 (Lucroy) and well over 2,000 for whipping boy no. 1 (Doumit). And they received like the Lucroy and Doumit we know in the minors as well, with 41 prorated runs saved and 38 runs cost, respectively.

Now let’s look at some catchers with less-established framing reputations.

At 33 years old, Erik Kratz doesn’t qualify as a prospect, but until a recent knee injury, he was having his first season as a regular in Philadelphia, benefiting from Carlos Ruiz’s suspension and hamstring strain. After two months of play he was fifth in framing runs saved. His ability to gain extra strikes for his pitchers has always been evident in his minor league numbers, with multiple seasons in double-digits and not a single year below average. In his minor league career since 2005, he has averaged roughly 20 runs saved per 10,000 pitches.

Derek Norris was in the package the Athletics received from Washington when they let Gio Gonzalez go. Norris had already shown good framing numbers at Double-A Harrisburg, where he saved an estimated 17 runs (which translates to 46 when you prorate it to 10,000 pitches caught) for the Nationals affiliate. He split the 2012 season between the Sacramento River Cats and the parent club in nearby Oakland and continued to post good numbers at both levels (42 prorated runs in Triple-A, 23 in the majors). This year, he was in the MLB top 10 as of the end of May. The saber-savvy A’s may have been aware of Norris’ framing performance when they had the Nats include him in the trade. But did Washington’s front office know that they were giving away a catcher who could steal a strike or two for their crop of young pitchers?

Norris was the top minor league framer in 2012. Behind him was Rene Rivera, then playing in the Twins’ farm system, who was signed to a minor league contract by the Padres over the offseason. His bat doesn’t seem to be big-league quality, and at 29, his chances of improving are slim. However, since we are starting to appreciate how valuable framing is to teams (look here for a skeptical view and here for a rebuttal), someone might offer him some time in the sun.

At no. 3 was another 29-year-old, Wilkin Castillo, who has not been deemed worthy of an entry in a Baseball Prospectus Annual since 2011. His scouting reports used to praise his defense—score another success for the framing stats—but his bat has kept him from reaching the majors. He currently plays for Vaquero de la Laguna of the Mexican League.

To round the 2012 minor league top five, we have two players in the Pirates’ organization, both still at a safe distance from age 30. Ali Solis, 25, who had a cup of coffee with the Padres last year, lacks offensive skills, as scouts see neither pop in his lumber nor a good approach, but he has a very quick release when attempting to gun down baserunners. Scouts also regard him as decent at calling a game, which might be marked as another point of agreement between their point of view and that of statistical analysis. Fomer first-round pick Tony Sanchez, also 25, gets positive reviews from scouts for his skill at receiving pitches, but like Solis, he won’t be carried to the big leagues by his bat. He could project to be a defense-first regular if he improves on his inconsistent catch-and-throw mechanics.

I’d like to throw in another name on the positive side: J.R. Murphy, who’s having a strong offensive season for Double-A Trenton in the Yankees organization, ranks 10th by my algorithm. There are a couple of reasons for listing him: he is very young (22), and scouts have negative notes on his framing skill (“Rough receiver… Doesn't always handle quality breaking balls well… letting the ball carry his glove on occasion and not helping his pitchers…”).

Of the five catchers at the bottom of the minor league framing leaderboard for 2012, four—Omir Santos, Jason Jaramillo, Hector Gimenez, and Anthony Recker—have already spent some time in the majors. Santos got into a game with the Indians in April and is now at Triple-A Columbus, and Jaramillo played for the Pirates from 2009-11 and now toils at Triple-A Tacoma. Gimenez is backing up Tyler Flowers with the White Sox, and Recker is backing up John Buck in Queens. Again, the minor league numbers are backed up by the big league stats: according to the PITCHf/x method of quantifying framing, the quartet had cost their teams over 50 runs in over 16,000 career called pitches caught in the majors through the end of May. The only one of the five 2012 trailers not to have made his big-league debut—he ranked second-worst in the minors last season—is Rossmel Perez, a light-hitting 23-year-old Venezuelan in Double-A for the Diamondbacks.

While the numbers I computed concur with the scouts on many players, instances of disagreement (like Murphy) are bound to occur if one systematically compares evaluations of catchers through two different lenses. That’s not a bad thing: as more established sabermetricians have stated, a stat that only confirms what we already know is not very useful.

Thanks to Zach Mortimer and Mark Anderson for providing information on the scouting side.