Baseball Prospectus - Chat with Aaron Schatz

Chat: Aaron Schatz

Welcome to Baseball Prospectus' Wednesday January 05, 2005 2:00 PM ET chat session with Aaron Schatz.

Aaron Schatz, the founder of FootballOutsiders.com, will be the lead author of Football Prospectus 2005.

Aaron Schatz: Greetings again, baseball and football fans. Once again I would like to thank Baseball Prospectus for allowing us to use their chat setup for this little discussion about America's other national pastime. For those who don't know me, my name is Aaron Schatz, I run a website called FootballOutsiders.com, and we do intelligent analysis of the NFL that is the equivalent of what BP does for baseball, except that our science is much more in its infancy. I've noticed in the early questions that there aren't too many playoff-related queries, so I do want to invite anyone to ask a specific question about the 12 teams in the postseason. But I'll take general questions too. I've got some 12 Rods rocking the box in honor of our favorite back-it-in playoff team in Minneapolis, so let's start talking...

denny187 (Hartford, WI): baseball is obviously the most individual team sport so it makes sense to be able to arrive at value based on statistics. football however is so team dependent. blocking is as important to the running game as the running back, quarterback decisions are as important to receivers as getting open, and play-calling probably trumps everthing. is it really possible to find value in statistics that are so team dependent?

Aaron Schatz: I'll start with this common question asked of Football Outsiders. The issue here is: What are we trying to accomplish? Are we trying to gain insight into how NFL teams can win more games, and what to expect from teams in the future? Or are we trying to gain perfect insight into how NFL teams can win more games, and absolute knowledge of what to expect in the future? To me, the first goal is a worthy goal. No, football analysis does not create clear player vs. player distinctions the way baseball analysis does, at least not at this point in its evolution and perhaps never. But we do think we have insight on the NFL that goes further than the conventional wisdom of other analysts. One of my mottos is that the best is the enemy of the better. We seek the better, and understand that we're not the best that could be theoretically possible. That best will never be achieved. Not even in baseball, just ask anyone who tries to analyze defense.

mgknig01 (Kentucky): After Clinton Portis' dreadful season, will GMs finally wake up and realize that, for the most part, offensive lines make great runners and not vice versa?

Aaron Schatz: I must admit that personally I was shocked by Portis' struggles in Washington this season. I wrote an article before the season pointing out that, when you compared Denver running backs all behind the same line, Portis was clearly the best since Terrell Davis. And the Washington rushing attack wasn't that bad in 2002-2003, contrary to popular belief. How great runner combined with underrated o-line became garbage is really curious. I think there is something to be said for the idea that Gibbs' blocking schemes don't mesh with Portis' style. One of those things we can't measure with stats yet.

That being said, I think that GMs are already beginning to understand that running backs are far more interchangeable than people realize. Look at how RBs have dropped in the draft in recent years. I seriously doubt that Alexander will return to Seattle despite nearly winning the rushing title. The bigger question is, when will the national writers and television analysts wake up and realize that offensive lines make great runners and not vice versa?

Mike (Burlington): Aaron, this may be on the list but I was wondering if you had ever considered looking at getting better ways to evaluate placekickers? For one thing, an adjustment for average attempt distance should be included. In addition, the effects of temperature and wind speed (though wind might be the toughest) should be considered. It would be great to see if this would drastically alter the ranking of kickers and also to see if there truly are "bad weather kickers".

Aaron Schatz: Oh, one right in my wheelhouse. We in fact have done exactly this. Our field goal measurement compares each kick to the NFL average at that distance. Our kicker measurements equally consider kickoffs and field goals. And both measures factor in weather and altitude, although in a more general fashion right now until I have more specific weather data for the last few years of games. You'll find a more specific look at kickers here:

http://www.footballoutsiders.com/ramblings.php?p=1798&cat=1

And a more general explanation of our methods here:

http://www.footballoutsiders.com/ramblings.php?p=1934&cat=1

Using a baseball analogy, as I like to do when chatting on BP.com, kickers are much like relief pitchers. They are generally interchangeable, they are inconsistent from year to year unless named "David Akers," the stats that measure their performance do a terrible job of really telling you who did the most to help his team win, and their performance in the clutch, no matter the stats involved, will stay in your memory forever. Adam Vinatieri may be the Mariano Rivera of kickers.

R.J. (Washington, DC): Does anyone have a chance of upending Philly in the NFC playoffs?

Aaron Schatz: Oh, yes, absolutely. First of all, anyone has a chance of doing anything in the NFL, where every playoff series is one game. Imagine the question: "Could the Devil Rays beat the Red Sox today?" Even though Boston is a far better team, the answer is of course yes.

I wrote about this in my DVOA commentary this week, but I would be worried by the recent performance of the Eagles. Not their play in the final two games, where they rested their starters, but their narrow victories in Weeks 14-15 against poor teams in Washington and Dallas (though Washington, at least, has a very god defense). And the fact is that the T.O. acquisition transformed that team in a way no acquisition of a wide receiver ever had before, and missing him is a huge deal. Their run defense weakness, particularly if you can stay away from Jeremiah Trotter, is going to be a problem if they play Atlanta. Despite the fact that Philly was the only NFC team to finish in the Top 11 of our DVOA ratings, and they are still the favorite in that conference, they have a much smaller chance of making the Super Bowl than they did four weeks ago.

james (new york): how many years of data do you have for dvoa.

Aaron Schatz: Right now 2000-2004 are up on the website, and I have 1999 finished as well. I have gamebooks for 1998, most of 1997, and most of 1996. Once I can fill those gaps, I can do those seasons, and I have someone in the NFL trying to get me data from prior to 1996. You can expect to see these years show up with commentary during the offseason, though probably not until June or July. Many thanks to Jim Armstrong for providing me with gamebooks from past seasons.

roone (Chicago): Prior to yesterday's Orange Bowl, do you know if anyone had calculated what the BCS rankings would have been this season using last year's rules? Also is there a football equivalent of SABR? Thanks and it's great to have you on BP!

Aaron Schatz: It is great to be here. I sent question one to our main college football writer, Russell Levine. He tells me that he read somewhere that with last year's rules, the final would still have been Oklahoma-USC, but unfortunately he doesn't have a link for you. Regarding your second question, there is the Professional Football Researchers Association. However, it should be noted that their primary thrust is football history, with very little work done on statistical analysis. Their website is: http://www.footballresearch.com/

johnsobs (Denver, CO): Have you done any research on how performance, specifically at the quarterback position, changes in the playoffs? Can players be quantifiably called "clutch?" Or is it like baseball, where data doesn't backup the claim?

Aaron Schatz: On the to do list. As I've mentioned before, that's a very long list, so there's no saying we'll have a chance to research it in the near future. If you think that small sample size is a problem with measuring playoff performance in baseball, though, try football. I mean, Peyton Manning has played a grand total of SIX playoff games in his life so far.

As far as clutch performance during games, I will say there is one major difference between baseball and football. At the end of a football game, the rules change, because you do not have an unlimited amount of plays in which to close a gap in the score. Therefore, a quarterback with clock management ability is important when it comes to winning games in the final minutes. Tom Brady is clearly better at that than, say, Quincy Carter. I suppose that could be described as a "clutch" attribute although I think of it as a separate issue.

mgknig01 (Kentucky): Which underdog will shock the pundits and win their first round game?

Aaron Schatz: Mike and I are working on some pretty intense playoff previews as we speak, which will appear both on our site and in the New York Sun newspaper (where, incidentally, BP also writes the baseball playoff previews). Here are my early thoughts, however:

San Diego and the Jets, over the course of the season, were really fairly equal teams. As a Jets fan, however, my worry would be that Chad Pennington is clearly still injured and it has become a major issue as far as his accuracy and ability to throw for distance. Meanwhile, the Chargers are extremely strong vs. the run, and that neutralizes the strongest aspect of the Jets offense. (If you enjoy good Chargers-Jets talk, check out the comments on the final DVOA commentary, which are very critical of my unusual ranking of the Jets offense higher than the Chargers offense, and the Chargers defense higher than the Jets defense.)

Minnesota and Green Bay is just wacky because my numbers say Minnesota has been the better team over the course of the season (MIN 13, GB 21) as well as the better team given recent performance (MIN 13, GB 16). In fact, Minnesota is the only NFC team that ends up with an above-average DVOA rating over its final four games, but they lost three of them and nearly the fourth. Whatever I'm still not taking into consideration in the formula -- as I said, it is still imperfect -- well, the Vikings are doing that. Given their struggles on grass and in cold weather, I don't see them winning.

Denver has a legitimate shot to beat Indy even though Indy is clearly the better team. Denver's pass defense has been very good this year in games where they were not on national television. I have no idea why appearing on national television makes Champ Bailey lose his ability to cover but I chalk it up to random chance. Denver's running game is perfectly decided to keep this game close. And Jake Plummer is ridiculously inconsistent, but inconsistent teams are much more likely to beat better opponents than consistent teams because they might just play their best game of the year that week without anyone expecting. That being said, I expect Indianapolis to win.

Seattle-St. Louis I have no idea. Seriously the Rams are ranked 30th -- THIRTIETH! -- in DVOA. This is like the Expos making the playoffs. Sorry, "Nationals." And they really have problems on the road. But they seem to be so deep inside Seattle's heads that I can't tell what is going to happen. The Rams, odd to say, are the most likely winner of the first week's road teams.

dukeandduke (Chicago, IL): What is your take on the health of the Pittsburgh Steelers coming into the playoffs and do you think they will take it all come February? Who will they face if they make it? Green Bay, Atlanta or Philadelphia?

Aaron Schatz: So let's move on to the real teams, the ones with the byes. Unfortunately we don't have a health expert on staff at FO but we do have Will Carroll, who can extend his knowledge of baseball injuries to football injuries. He takes part as special guest in our playoff draft and roundtable and I asked him about Ben Roethlisberger's ribs. He said it will hurt, but not prevent him from playing, it is all about Big Ben's pain tolerance, and the bye is a huge help to give him the rest. I asked our resident Steelers fan Ryan Wilson for his take on the defensive injuries. NT Casey Hampton is out, but OLB Clark Haggans, ILB Kendrell Bell, CB Chad Scott and CB Deshea Townsend should all be ready in two weeks. If Haggans or Bell aren't fully recovered from groin injuries, ILB Larry Foote and OLB James Harrison have both done a solid job in their place. But one of the hidden effects of the injuries is that moving backup linebackers like Haggans and Foote into the starting lineup has taken them off special teams, and this was a major issue in the close win over the Giants.

Back to my opinion on the Super Bowl. DVOA says the Steelers, Patriots, and Colts are all about equal in ability, but I have a hard time thinking the Colts can win twice on the road given the history of dome teams in cold weather. I'd say a Steelers-Pats AFC championship would be a 50-50 shot as to who wins, and the winner would clearly be greatly favored to take the Super Bowl championship. In the NFC, I've expressed my worries about the Eagles. My numbers say the Falcons are very overrated and what is interesting is that they are perfectly designed to take advantage of Philly's weakness on rush defense but not Green Bay's weakness on pass defense. Like I said, the Eagles should still be favored, but they are certainly not guaranteed passage to Jacksonville.

Andrew Brecher (Arlington, VA): How important is home-field advantage in the playoffs, vs. the regular season? And can you isolate the effect of HFA per se vs. the effect of the home team (usually) being simply the better team?

Aaron Schatz: Whoo-hoo! Brown University Economics Majors Class of 1996 are in the hizz-ouse!

I haven't done extensive work on home field advantage, but it does exist and gets stronger the later in the season you go. It is also true that warm-weather and dome teams in general perform below their usual level when playing in cold weather from November on, particularly when the game starts at 4pm (local time) or later.

When it comes to the playoffs, the most important aspect to remember is not the effect of the home field but the effect of the bye week. Home teams (who were, of course, better during the regular season) are only 13-11 in the last 12 years of conference championship games. But getting the first round bye is critical. Since the playoffs went to six teams per conference in 1990, home teams are 45-11 in the second round.

e (Chicago): This past October, one of the most popular features of Baseball Prospectus was the Playoff Odds report. Although your football statistics haven't evolved as much as baseball's, would it be possible to get even a rudimentary playoff odds schedule on your site? Could be a good source of new views =) PS. Congrats on your www.twominutewarning.com win! 1-4-0 vs 0-4-1? A win is a win...

Aaron Schatz: First of all, I should point out the typo -- I meant that the Denver running game is DESIGNED to keep this game close.

I loved those Eels records, E. Your congratulation and your question are actually connected. Yes, I won the Two Minute Warning Head-to-Head Pick Contest this season, despite going 1-4 in the final week (I won my first three rounds going 4-1 each time). Despite this, DVOA is not designed to specifically tell you which team is more likely to win a particular game, as it does not take into account a number of issues that are game-specific as opposed to season-general (don't know if I can use that as a description, but you get the idea). A few readers have been using DVOA to pick games each week, but I often stress that their efforts are "unofficial."

Because I can't say that DVOA of X playing at DVOA of Y equals Z% chance of winning, I'm not quite ready to put together a playoff odds report that would be statistically sound. But it is a great idea and I'm hoping that we can do a feature like that for next season.

saveferris (Baltimore, MD): Aaron, research presented on your site has indicated that there is virtually no year-to-year correlation in fumble rates at the team level. Do you believe that this may also apply at the individual player level and, if so, do you have any plans to study this issue? Many players have the reputation as fumblers, particularly RBs, yet I have not seen any objective study of the issue on a rate basis (it�s always raw totals), on a comparative basis, or with year-to-year correlations. Obviously, if it were found that fumbling (or not) is not a skill most players posses, then the possibility of a market inequity becomes very real � players who are perceived as fumblers may be undervalued, while those with the perceived ability to avoid turnovers may be overvalued.

Aaron Schatz: I've done some research here, but the sample size for players is so small (seriously, how many payers fumble more than 6-8 times a season?) that it is really unreliable. However, it does seem that there are players who have the skill of holding onto the football more than other players, even though the statistical noise is colossal. And there is, I believe, year-to-year correlation in fumble rates at the team level. But there is not year-to-year correlation in FUMBLES LOST or FUMBLES RECOVERED at the team level. The skill is knocking the ball loose. Recovering said ball is in nearly all cases a totally random event dependent on where the ball bounces and who happens to have possession of it when the refs finally get to the bottom of the pile. (And, as Jason Beattie often points out, who is pointing more strenuously.) This is why our numbers now treat all fumbles as equal value, no matter who recovers. Instead, the value of a fumble is determined by the percentage chance that a fumble of that type is generally recovered by the defense: fumbles after long pass plays being highest, followed by short pass plays, rushes, sacks, and then aborted snaps (if I am remembering correctly). Our numbers began to correlate much better from game to game and year to year once we started treating all fumbles as equal value.

will (new york): have you done any studies on homefield advantage? Do you use homefield advantage in dvoa? if you don't, do you plan on using homefield in dvoa?

Aaron Schatz: I think I've answered this in the last couple Football Outsiders mailbags, so it is a popular issue. The answer is:

1) Yes, HFA is real, it makes up a difference of roughly 17% between two teams, and it seems to get stronger in the final eight weeks. Also, no team seems to have a greater home field advantage than any other that remains constant from season to season. I really question this result because in every other sport Denver has a bigger HFA than any other team, and why would this not be the case in the NFL? Plus, non-DVOA research indicates that cold weather teams have a greater HFA, so I'm guessing DVOA research will eventually indicate the same once I have more years to work with.

2-3) Not yet, but I'm looking into it for next season.

geer08 (Birmingham, AL): As a Redskins fan, believe me, I was disappointed by Portis's season...but to call it dreadful is going a bit far, yes? He finished 8th in rush yards for the season, and was held out of several of the last few games, so it's simply his touchdowns which were lacking...don't let his ONE season away from Denver be an indictment of all GMs and future RB acquisitions.

Aaron Schatz: His DVOA plummeted from 15.2% to -11.2%, so on a per-play basis he was acheiving far less success. But he did seem to be better in the second half of the season, so we'll see about next year. Which brings me to our last question before I go...

charlesford (Arlington, VA): Which teams that didn't make the playoffs do you see as having the best chance to crack the playoffs next year? Was there a team that had a particularly brutal schedule that might see a lot of improvement just by playing a league average schedule?

Aaron Schatz: Yes. Imagine if Buffalo could play half NFC teams instead of only four NFC teams!

A couple weeks ago for a mailbag I ran some early 2005 projections for offenses, but I have not done it for defenses. Therefore understand that the comments I am about to make are not completely grounded in analysis of each team's specific splits for the 2004 season. Let me point out four teams right now, based on different issues.

1) Buffalo. Teams that finish higher in WEIGHTED DVOA than in TOTAL DVOA have a tendency to improve the following season. This defense is fantastic (although remember, offense is generally more consistent from year to year than defense is). They clearly would benefit from improvement at one position, quarterback, and Willis McGahee can be expected to improve in his second post-ACL surgery season. The biggest problem for them is that the AFC is so good, and their division is so good. Speaking of which, team two is...

2) Miami. The defense is still strong, the receivers are good, the quarterback is probably fine. This team is six players and a new coach away from 9-10 wins again next season. Those six players: RB and an entire offensive line.

3) Cincinnati. Remember what I wrote about offense being generally more consistent from year to year than defense. Their offense got better as the year went along, their cornerbacks are very underrated, now they just need to get better in the front seven.

4) Washington. Maybe we just picked the wrong year to predict that Gibbs could turn this around. As I noted earlier, Portis improved in the second half, as did the entire offense once Mark Brunell was no longer the quarterback (boy, did he fall off a cliff quickly). Washington also has an advantage that all NFC East teams will have and that the AFC East and NFC South had this year. I don't see the four NFC West teams being any better next season, and they are on the schedule for Washington and their other divisionmates (as well as the AFC South).

Aaron Schatz: Whoops, let me clarify. I mean that the AFC South gets to play the NFC West also, not that the AFC South is also on Washington's schedule. Thanks again for reading, and please come visit Football Outsiders. Once again I express my gratitude to the BP folks for lending us their space. I may or may not be back before the Super Bowl, depending on how much the baseball coverage has ramped back up by then. Otherwise, I'm sure you'll be hearing from me in this space when we are ready to bring out Pro Football Prospectus 2005. Until then, good luck to all your teams in the playoffs, unless they play the Patriots.