Fantasy Beat

I still haven't seen any actual reasoning or explanation for some of the ludicrous projections that were put out in the late February update. Chris Davis had a 90th percentile of 52 homeruns. Alex Rodriguez had a 90th percentile of a .700 SLG%. These and many many more questionable numbers were changed with the recent updates without any mention of cause. These errors (yes, I'm calling them errors because they were so out of whack with what PECOTA used to spit out) were not a playing time issue. They weren't a 10-year issue. They were severe problems in the one-year calculations for a bevy of players.

In other words, PECOTA was worthless weeks ago, and for us -- the subscribers -- to have any sense
of it having been reconciled, it would be nice if you guys would actually fess up to what in the world you did to screw these numbers up so badly and what you did to fix them. Sweeping it under the proverbial rug would not be a good first step, I would think.

I think it's telling that in 2003, when I was first introduced to the site, Baseball Prospectus told me everything I needed to know about statistics and value and the great world of sabermetrics in baseball. Now? I only come here for prospect reports phoned in by scouts. I think the message has been lost somewhere, gentlemen.

So. . . what is the new hitter card build intended to fix?

We're all paying for this stuff, Dave. I understand that BP is going through growing pains trying to make the system better and more efficient. And I'm confident that once this is over, we're going to end up with a better PECOTA. I get that.

But in the interim, some of us are entering fantasy drafts with hundreds of dollars in entry fees. And we pay BP for a quality product. I appreciate the updates, but, at minimum, an acknowledgment of what needs to be fixed would be appreciated. We've all figured out on our own that it has to do with outlier projections and multiyear projections, but an acknowledgment from BP as to what on Earth is going on would help out your paying customers.

One of the problems with this process has been the perceived lack of transparency. You come onto the blog and tell us you're working on it, but you don't say what the problems are. You owe it to paying customers to give us a little more detail.

Thanks,

Chris

Can't wait for the team-by-team article. When Clay published the last one, he said the remaining teams would be ready the next day! Definitely eager to see that. It was one of the best pieces I've seen on BP in a while. Any specific ETA?

I agree that transparency is the issue here. You guys clearly are still tinkering with PECOTA. Many of the projections on the weighted means sheet are still moving. Between 3/16 and today, virtually every pitcher's rate stats have changed, and many of them have changed dramatically. To pick just one example, Scott Baker's innings pitched haven't changed, but his projected ERA has moved from 3.92 to 4.11. So, you're making other adjustments to the projections, and it would be nice to know what they are.

One thing going on. Things are backed up because they badly underestimated how difficult and time-consuming it would be to switch over from Excel to whatever platform they've chosen. They've said this already.

Second thing. The reason the PECOTA cards have gone to the Beta testers is that, so far as BP knows they are now ready to go. First they'll see whatever (hopefully minor, data misentry type) things might be wrong with them.

I think you guys are mistaking 'tinkering' with 'fixing'. It's clear that some aspects of PECOTA were not working properly at its first port out of Nate's coding, or even its 2nd or 3rd. Most of those seem to be fixed now, and the book projections have always been in line with what's going up on the website.

If you're not a keeper-league player or someone who's drafting minor leaguers, there's nearly nothing you're going to get from the new hitter or pitcher cards that you don't already have access to.

I think the transition issue has had less to do with platform and more to do with the intricacy of the system that used to be maintained by one person who designed it (Nate Silver) and now not only has to be ported, but had to be learned in the process by the people doing the porting.

It certainly sucks to feel like PECOTA is less than the most reliable rock you have available to lean on that it's been in the past, but I do think the histrionics are excessive: if you have access to the book projections, only a significant minority of fantasy players do not have access to the full data they need to draft this weekend.

Those of you who are pushing should keep in mind that the situation is as it is: an early release is just more likely to include bugs or problems or simple miss-links or accessing of old databases instead of the newer versions. Would you take a release today in exchange for a potential major database error, or a monday release without one?

Let's step back for a moment:

Do many of us love and support BP, and rely upon its PFM and PECOTA for our drafting? Yes
Do we sometimes look at a projection and think, "that doesn't look right" (either good or bad), and discount it from our draft sheet? Yes
Do we also seek out other fantasy draft projections at the same time? Yes

Is $40 for a year of all services a good deal, or $20 for fantasy features only a good deal? Yes

Has BP botched the normally smooth rollout of the PECOTA? Yes
Is this the first major glitch most of us have experienced in our dealings with BP? Yes
Do customers have a right to be disappointed/angry about this? Yes

BUT .... let's realize that there has been a lot of personnel turnover at BP during the last year, and they're trying to automate an Excel process featuring tons of data and inter-related algorithms.

Just my opinion here, but if you are playing fantasy sports for "thousands" of potential dollars, perhaps you've lost sight of the "fun" that is supposed to be involved.

Fantasy sports are supposed to ADD to one's enjoyment of the game. Spewing vitriol at a supplier of information used in fantasy gaming may be cathartic for the venter, but after a certain point, it only turns into static.

Like some of my favorite players, I'm willing to give BP an "off-year", and I'm willing to pony up another $40 to give them another chance.

Comeback player of the year, 2011 perhaps?

My biggest complaint with the 2010 hitter cards -- the lists of comparable players -- now appears to have at least been partially fixed.

Everth Cabrera no longer appears to be a Hall of Fame caliber player (although his #6 comp is still Hanley Ramirez, which is kinda of fun to think about).

Matt Weiters' comps, sadly, are still a bunch of guys who never really had career as MLB starters. Would love to know why PECOTA "thinks" this.

When the release is done, could you guys offer some commentary on the more controversial projections/comps?

As a subscriber who gets well over $40 a year in value even without PECOTA, I'm not about to bail.

Regarding PECOTA, I understand it will always be subject to improvement, and fully support that it continue to be improved. In my opinion, though, it's simply timing. That said, here is my expectation / proposal:

- Continue to work on improvements during the year, but with a clear "stop tinkering/improving" date that is WELL before season's end (e.g. Stop tinkering in October). And for gosh sakes, take full advantage of those volunteer beta test guys. A group that I would be more than please to be part of.

- Once the 2010 MLB season is over, simply add the 2010 data to PECOTA, knock out the FINAL version of the cards, and know that BP customers everywhere will rejoice.

- Repeat the cycle annually.

Although the above is clearly simplified, I think it makes the point.

I understand that PECOTA will never be "good enough" as long as there are brilliant minds with ideas that will improve upon it. But a very good product that is timely is much more valuable than an "even better" product that doesn't reach the shelf in time.