July 9, 2013

Baseball Therapy

An Attempt to Measure Makeup

by Russell A. Carleton

the archives are now free. All Baseball Prospectus Premium and Fantasy articles more than a year old are now free as a thank you to the entire Internet for making our work possible. Not a subscriber? Get exclusive content like this delivered hot to your inbox every weekday. Click here for more information on Baseball Prospectus subscriptions or use the buttons to the right to subscribe and get instant access to the best baseball content on the web.

Subscribe for $4.95 per month Recurring subscription - cancel anytime. Subscribe for $39.95 for one year a 33% savings over the monthly price! Purchase a $39.95 gift subscription a 33% savings over the monthly price! Already a subscriber? Click here and use the blue login bar to log in.

Time to put on my other hat. Let's leave the wonderful world of sabermetrics behind for a few minutes and venture into the world of psychometrics. In one of my former lives, I conducted (neuro)psychological testing, mostly on children and adolescents, as part of my graduate school requirements. I'm proud to say that I administered the Rorschach only once. Because they made me.

Psychological testing isn't nearly as scary as it sounds. Most of the time, it was a question of finding out more information about a diagnosis (is there a learning disorder present?) or differentiating between two possible diagnoses (is this ADHD or is the problem with attention due to anxiety?) Generally, we stuck to tests that are very well-grounded in research and that measured specific behaviors or cognitive skills. In fact, the process of writing up an evaluation of a psychometric battery of tests has the same feel as writing up a sabermetric view of a baseball player. It's very numerical and grounded in previous research—and if you don't know how to use those numbers properly, you'll come to some very wrong conclusions.

In my professional work, I did a lot of evaluations around issues of learning and attention. Often, the patient was having trouble with learning in school, and it was my job to figure out why. By extension, it was also my job to figure out what could be done to make life easier for the parents, the teachers, and most importantly, the kid.

But like so many things in life, I found myself asking "How can I apply this to baseball?"

Majoring in Baseball
Baseball develops its young players a little differently than the other "major" sports, or at least football and basketball. There is no "minor league" system to speak of in the NFL, and a limited one in the NBA. Players emerge from each league's respective drafts and are generally expected to contribute immediately, even in some limited capacity. It will be a couple of years before most of even the highly regarded MLB draftees who had their names called a few weeks ago sniff the big time. In fact, in an odd sequence of events, a baseball player might be drafted at the age of 18, given a half million dollars, and—despite the fact that he's never lived apart from his parents—sent to a fourth-tier city a thousand miles from his home. If he's an international signee, it's entirely possible that he's 16 and being sent somewhere where no one else speaks his language. Now, he gets to learn how to play big league baseball. And become a man.

Young baseball players are an altricial species. It's rare that a player is ready to go directly from high school (or even college) to the majors without a little bit of instructional time, and often several years’ worth. That means that baseball scouts have a much harder job than scouts do in other sports. They have to look at an 18-year-old and, sometimes devoid of context, figure out what sort of talent he will have at age 27. That's like asking someone to predict what a room full of fourth graders will look like at their high school graduation. A gifted athlete can succeed on pure athletic skill at the high school level, mostly because he's going up against competition who might be there because they didn't make the bowling team. What will happen when he faces guys who know what they're doing?

What has always struck me as something in need of repair was the fact that the language of scouting is heavy on vocabulary for describing physical build and playing skills, but relatively light on what might be going on under the surface. The story of baseball is littered with guys who had immense physical talent and promise, were never hurt, and yet did not make much of an impact. Look through the recap of the first round of any MLB draft, starting about five years ago. You'll pick them out pretty quickly. What happened?

The word that's most often trotted out in this situation is "makeup." It's one of those vague words that probably has as many definitions as it has people who use it. There are connotations of players making poor decisions, lacking desire, and just plain being a jerk rolled into the term, mixed with a small dash of moral disapproval. All of the above are probably true from time to time.

What about the guys who honestly try hard, but still just don't make it? There's something of an assumption that players are all equally teachable and that guys who start out at 18 with the most skills will be the guys who have the most skills at 27. It's not a bad way to bet, but it never works out that way. Maybe there's something that we're missing? Some guys who have trouble learning? Some who have other issues going on? Maybe there's more to makeup than just a bad attitude. What if there were ways to at least screen for some of the issues that might pop up?

Recently, some teams have become more aware of these issues, and some have started screening potential prospects in their own ways with their own proprietary methods. In what follows, I'd like to take a slightly different approach. I propose to develop a battery of tests comprised entirely of measures that are publicly available and can be purchased (or are free!) that might be useful in explaining some of the slack in the system. Many of them are tests that I've used in my own work in a psychological testing clinic and all are well-known and well-researched in the psychological community. (They're admissible in court!) In the process, I'll highlight a couple of issues that we don't often think about that might explain why first-rounders end up as busts and fifth-rounders end up as All-Stars.

Attention, Reaction Time, and Impulse Control
I've written extensively about issues of attention and focus, and how baseball demands a great deal of both. Baseball, more than any other sport, requires constant vigilance often combined with limited action, and in amounts that strain even typically developed young men.

In the clinic, one of my favorite tests was the Conners' Continuous Performance Test (CPT). I primarily used it to screen for ADHD, but it can be used for much more than that. It lasts 15 minutes and consists of a series of letters that flash on the screen. The person being tested is supposed to push the space bar when a letter comes on the screen, except on one special letter. Over those 15 minutes, the rate at which the letters are presented varies, and the main outcome of interest is actually the person's response time. Does it get longer when the time between letters goes from one second to four? (Attention is wandering.) Does it get significantly longer as you get into the 12th minute of the test? (Sustained attention might be an issue.) Does the subject have trouble stopping himself from pushing the space bar in response to the forbidden letter? (Is there an impulse control problem? Will he swing at anything?) Of course, no one will be perfect at the test, but the rates are compared to the league... er, population norm. In addition to yielding information concerning attention, the Conners’ Test is also a decent indicator of simple reaction time.

If a player can't sustain attention, it will affect him not only on the field, but during instructional time. When the coach is showing him a different way to position his hands or some trick to use to get a better grip on his changeup, will the player be able to concentrate long enough to hear the coach's message? It's not that teams shouldn't draft these guys. It's that they might have to make a few extra accommodations to get the best results out of them.

Self-Correction and Cognitive Flexibility
One reason that a player might flame out is that he's not willing to make adjustments to his game. Playing against the bowling team's rejects in high school, he might have hit .570, and he might have a chip on his shoulder. Let's see what happens when he faces decent (much less really good) pitching. Will he drop the chip and accept coaching?

One off-the-shelf assessment that might give us some idea is the Wisconsin Card Sort Test. There's a version of it that includes actual cards, but this one is better done on a computer as well. The WCST starts off with purposely limited instructions. The person taking the test is given four cards with different shapes, colors, and numbers on them and then asked which of the cards best matches a fifth card. He gets no feedback on being right or wrong until he makes his choice. Once he gets it right, he generally tries the same strategy again (match the color? match the number?), and for a while it works. Part way through the test, mercilessly, the rules change and again, the subject is not told anything. Will he stubbornly persist with what worked before? Will he show a longer response time (thinking about it more?) when things change? Will he be willing to try a new strategy?

Like most psychological testing, this isn't an exact replica of what a player will be asked to do in his actual baseball work. He might pass the WCST with flying colors but still flame out because he stubbornly refuses to try something new in real life. But the WCST screens out those who lack the underlying cognitive flexibility to begin with.

Verbal, Visual, and Kinetic Learning Styles
Some people learn best when you show them how to perform a task. Others learn best when you explain it to them verbally, and still others learn best by doing. Some are fine hearing something in isolation. Some need context. When I assessed learning issues, one thing I always tried to get at was how the particular child I was working with learned. (This, by the way, is why one-size-fits-all developmental systems can be very counter-productive, and why a coach who can explain things in multiple ways is probably worth a couple of wins per year.)

There are several ways to get at this one. There are memory tests, such as the Wechsler Memory Scale (WMS), which present a person with verbal information (a list of words) or visual information (an abstract design) to remember and later reproduce. There are also tests of memory for various hand gestures (although not on the WMS). The full WMS takes a long time to administer, but it's possible to administer specific subtests to get just the information you need. In doing so, you can get a good idea of how a person best encodes and remembers newly presented information. That might prove handy when deciding how to present lessons on the training field.

Potential Mental Health Strengths and Challenges
Men are notoriously hard to get through the door of a psych clinic, and an untreated mental health problem can easily get in the way of a promising career, so maybe it's best to screen for any as-yet-identified mental health concerns. There are a number of good screeners both comprehensive (Adult Self-Report, BASC-2 System), and brief (Strengths and Difficulties Questionnaire). Ditto for alcohol abuse and other substance use problems (and there are several screeners from which to choose). Teams would want to make sure that appropriate services are available to the player both as a matter of player development and as a matter of being decent human beings.

Dealing with Life
On the flip side, teams might also ask questions about a player's strengths. When things get rough, how does he cope? Make no mistake, things will get rough. The player will deal with failure on the field, perhaps on a level that he's never even thought possible. And then there's the simple fact that the minor league experience is a giant transition in life, as Twins vice president of player personnel Mike Radcliff points out in Parker Hageman's BP guest piece today. We often forget that these are 20-year-old guys away from home for the first time, sometimes taking care of meal planning and schedule-keeping for the first time on their own, and perhaps entering into their first mature romantic relationships. In addition to becoming major league players, they are transitioning into the role of adults within society. That isn't easy, no matter what job you're working.

There are methods of coping that show better results (active problem solving) than others (ruminating about the problem), and prospects who seem to use the latter might end up unable to deal with the pressure of becoming a major leaguer or life more generally. There are several ready-made coping inventories (including this one) that are quick to use. In addition, teams might want to know whether a player has the skills to function as an adult in society. Does he know how to complete a rental/lease agreement? Can he balance his bank account? Does he understand how to have safer sex? If a player has trouble in these other areas, it might spill into his play on the field. Some teams have programs to help young players with these skills, but it might be nice to know in advance whether a player will need that help. The Ansell-Casey Life Skills Assessment might help here.

Does He Like Baseball?
Here's a question that doesn't often get asked by fans. Do the players on your favorite team enjoy playing baseball? There's a difference between being good at something and enjoying it. When I was preparing this article, I talked to BP's own Jason Parks, who had an interesting take on the subject. The player development process requires a great deal of commitment to be done right. There's a lot to learn, and many teams are very strict in their teaching methods, with some, according to Jason, approaching military strictness. Will the player be willing to put up with that in order to get a shot at a major league career? He'd better like baseball.

Prospects are generally chosen because they have a great deal of natural talent. But the key question is whether the prospect has the will to actualize his natural talent. (Maybe it would be best to shorten that to some sort of acronym?) No, there isn't a test for this one. It's something that you have to get some field reconnaissance on from someone who is skilled in having these types of conversations. Someone like… a scout.

The question "Is he a jerk?" might be good information to find out as well.

Tying it All Together
Maybe I'm a little biased, but I believe that a full understanding of the mental side of the game—more than just the "rah-rah, you've got to work hard!" motivational piece, but issues of neurology and psychometric assessment—represent a great unexplored area in the study of baseball. There are real differences between people in these areas, and it's fairly easy to understand how a problem in one of them might derail a promising career. My goal here is to show that these soft factors aren't big black boxes. They are actually well understood within a broader context. We just haven't had the chance to look too deeply into them in baseball.

The other message that I want to convey is that development (of any kind) is not a nice, steady linear progression. It's hard work, it happens in fits and spurts, and it isn't just about what a player looks like from the neck down. There are a lot of issues that can impact a player's development. Sure, teams bear some responsibility to screen for those issues (and some need to do more), but before you trash your favorite team's player development department or curse the name of that hot prospect who looks like he's a bust, remember that it's not as easy as it looks.

It's not that teams don't recognize this, and again, some are already addressing the issue. (If only they would release their databases!) There will be limits to how much outsiders can truly measure these effects, although I think we do ourselves a disservice if we don't at least try. But make no mistake, there is plenty to be discovered here (and I left even more issues that I wanted to discuss on the cutting room floor). If you're a baseball researcher who’s begun to believe that everything about baseball that can be discovered has already been found, may I suggest that you poke around this area?

Russell A. Carleton is an author of Baseball Prospectus. Follow @pizzacutter4
Click here to see Russell's other articles. You can contact Russell by clicking here