14 comments have been left for this article. BP Comment Quick Links
| |
|
|
May 13, 2016 Tools of the TradeTesting the Axe Bat's Claims“Soon after I started hitting with the Axe Bat, I realized that the old bat had to go, because it was now obsolete.” – Damian “I love the way the Axe Bat feels, and regular round handles feel horrible now. It's really helped me become a better hitter; the data is clear.” - Carrington A few months ago we introduced you to the Axe Bat after talking to Hugh Tompkins, their Director of Research and Development. During that interview we dove into the concept behind the bat, how it came about, and the unique technology behind it. Since then, the Axe Bat has been popping up all over baseball, including in the hands of some of the game’s best hitters. I won’t dive into the details about the Axe Bat and its origination; I recommend reading our interview with the Axe Bat team for more background on that. What we will discuss today is some of the claims made by the Axe Bat marketing team. The claims in question are pulled directly from their website: The Axe is the first and ONLY bat on the market that's designed to fit your hands. Recent studies confirm that the Axe: 1) Is more comfortable 2) Delivers a more efficient power transfer 3) Increases bat speed through additional barrel acceleration 4) Reduces hamate bone/ulnar nerve injuries and incidents of thrown bats To date Baden Sports, the parent company of Axe Bat, says it has backed up these claims through ergonomic and biomechanical research. The most extensive study on their product, completed by a team at UCLA, exemplifies much of their support for these claims. You can read the results of that study as it applies to the claims above on their website here, and you can also read more details from the full study here. We wanted to take it a step further though, performing an independent study in a real world setting. Specifically, we wanted to look at whether the Axe Bat stood up to the performance claims that they make.
About K-Zone Academy/HitTrax
HitTrax is a data capture and measurement system that uses infrared technology to measure and then simulate the performance of a given pitch or batted ball. The concept is similar to a golf swing simulator, which many of you are familiar with. The system also stores data, allowing us to make direct comparisons between a player’s sessions over time, a key component of this particular study. Experiment Design A baseline HitTrax session was recorded for each player using a conventional wooden bat. All players received maple Axe Bats matched to each hitter's preferred length and weight. Using wood bats ensured that any improvements in performance could not be explained by minute differences in bat construction, sweet spot, etc. since there is less variation between wood bats than their composite/alloy counterparts. All players used matching maple pro-model bats, exactly alike in all ways including length and weight. The only difference was, of course, the design of the handle. There may have been small performance differences between the bat because of small, unpredictable variations in wood grain or density, but those differences should be largely imperceptible in terms of performance. There is a learning curve associated with the Axe Bat, so at the manufacturer's recommendation, the players went through a two-month adjustment period with the Axe Bats. Players competed in “hitting leagues” at the K-Zone Academy facility over the winter. The HitTrax system captured exit velocity, projected distance, pitch speed, and launch angle, and assigned each hit a point value based on its trajectory. After the two-month adjustment period, each hitter came back for a few sessions where they alternated between the Axe Bat and a conventional wood bat. It's worth noting that the participants in this study were aware of the product claims. The claim that we're going to primarily focus on (because it's the most straightforward to measure) is whether the Axe Bat increases bat speed. There are a few devices based on inertial sensors that measure bat speed, but—skipping the #GoryEngineering—the unusual knob shape means we couldn't use any of them. Still, the HitTrax data gives us information about how the ball leaves the bat that we can work backward from. Here are some of the key metrics that we used to analyze the performance of the Axe Bat versus our controls:
Determining Significance The power of a T-test with nine players would be minimal at best, so after consulting with Russell Carleton, we also created linear mixed models to quantify the effect the Axe bat had on each metric from the HitTrax system. (For the technically minded, our model had bat type as a fixed effect, and player as a random effect.) The mixed model lets us control for the different skill levels of our subjects. Here’s a sample of the data to give you an idea of the talent level of the players involved in the study[ii]:
Eight of the nine hitters in the study saw their maximum exit velocity come with the Axe Bat, but of course we need something a little more rigorous to see what effects, if any, the Axe Bat had. The Results Our mixed model gives us similarly tepid results. The model suggests that, given a random player from our sample, we would expect him to have an average exit velocity of 74.4 mph with the control bat, and 74.9 mph with the Axe Bat. The p-value associated with this model[iii] is 0.19, also above the level of significance. The graph below shows the average exit velocity for each hitter with the Axe Bat (in blue) and a control bat (in orange), with error bars representing the standard error around each hitter’s average.
But remember what we're trying to test: Baden's claim is that the Axe Bat improves bat speed, not exit velocity. And exit velocity depends not only on swing speed but also quality of contact: No matter how violently you swing, the ball's not going anywhere unless you catch it with the sweet spot of the bat. We also examined the average distance for each player, under the assumption that if the Axe Bat “delivers a more efficient power transfer,” the result should be an improved average distance. (For this test, we removed groundballs and pop flies from our sample, taking only those hits with a launch angle between 0°and 45°.) And in fact, six of the nine participants hit the ball further with the Axe Bat on average than they did with the control. Over our nine-hitter sample, the average distance increased by 5.7 feet, though our paired T-test (p = 0.24) did not suggest significance. When we account for the random effect of the player, the data became a bit more suggestive. For a random player from our sample, the average distance with the control bat was 191 feet, compared to 196.7 feet for the Axe Bat. The p-value associated with this model was much closer to the typical significance level (p = 0.11), suggesting this effect could merit further testing. The graph below shows the average distance for each hitter; as before, the error bars represent the standard error.
It’s worth reiterating that this result isn’t statistically significant in our nine-player sample, so we should be wary of drawing broad conclusions. It merits consideration for a more robust study in the future. We’ve looked at average exit velocity as well as overall distance, but that still doesn’t help us answer the claims in a specific way. Not to mention that neither metric passed the standard 0.05 benchmark in the T-test. It would be hard for us to hang our hat on the results relying on just those two things. But HitTrax also provides a “hard hit” velocity—the average velocity of all balls that left the bat at or above 90 percent of the maximum exit velocity[iv]. By definition, these are going to be balls that were well struck, and thus represent the best proxies for bat speed. This is because the majority of these hard-hit balls inherently have a consistently higher quality of contact, removing one of the confounding factors we noted previously. The Axe Bat produced an average exit velocity on hard hit balls of 88.6 mph, whereas the round-handled bat average was 3 mph slower at 85.6 mph. Seven of the nine hitters in the study saw an increased hard hit exit velocity with the Axe Bat, and the p-value is a statistically significant 0.03. The mixed-effect model disagrees with the amount of the effect, but agrees that the difference is statistically significant. With the round-handled bat, the average hard-hit velocity was 86.3 mph; with the Axe Bat, the average hard-hit velocity was 86.9 mph. That difference of 0.6 mph doesn't look much better than the exit velocity for all hits discussed above, but the much smaller standard error of the difference (s.e. = 0.16) means that this model has an associated p-value of 0.00013. The graph below shows the average exit velocity for each hitter on hard-hit balls. The error bars represent the standard error around each hitter’s average. Note that Jared had one hard-hit ball with the Axe Bat, and thus standard error cannot be computed for that average.
With only nine players and a limited timeframe to put together this study, this result suggests the most strongly that a more robust study would be helpful to truly understand the impact of the Axe Bat on performance. Specifically, a study focusing more intently on Hard Hit Exit Velocity and fundamentally bat speed would be in order. Conclusion Swing speed is the primary driver of exit velocity, and our research suggests that the Axe Bat increases batted ball velocity on the whole, as exemplified by hard-hit balls. We also saw an increase in batted ball distance with the Axe Bat, though our result wasn’t statistically significant. There does seem to be some merit to the claims made by Baden Sports, but a more robust long-term study would be required to completely corroborate the claims. We asked Tompkins for his thoughts on the overall study and what these results might mean for the Axe Bat: These are exciting findings. Our study so far has focused on the biomechanical advantages of the Axe Bat and based on what we’ve seen there, we anticipated the likelihood of these kinds of results. Still, it’s very exciting to actually see the data and see some of the benefits quantified. I’m thrilled with the outcome and I think this is just scratching the surface of what can be done to learn more. The more players use the Axe Bat, the more we think they’re going to be able to take advantage of the benefits. The swing of the bat is so nuanced, and such a product of timing and skill, that you would expect the longer players use the Axe Bat, the better they would get at taking advantage of the design. *** There’s one other thing we should mention. The quotes at the beginning of this article came from two of the collegiate players that took part in this study. They weren’t alone in their praise for the Axe Bat. All nine players elected to continue using the Axe Bat after the conclusion of the study. While there’s no p-value for something like that, it sure says a lot about the impression the bat left on these players. Thank YousThere are many people to thank for their help pulling this study together. First and foremost, a huge thank you to Dan Kopitzke and the K-Zone Academy for actually performing the study, gathering the data, providing quotes from the players, and helping analyze/review the results. Another huge thank you goes out to the nine athletes who took part in the study and allowed us to make them guinea pigs for this comparison. They all took hundreds of swings changing bats at our whim without complaint, and for that we are very grateful. Thanks to Stephen Reichert who played a critical role in starting our conversations with Axe Bat, as well as providing a critical eye throughout the course of the study. Finally, thanks to Russell Carleton and Jonathan Judge for taking time to review and provide recommendations on analysis.
[i] Determined as balls with an exit velocity of at least 90 percent of that player's maximum exit velocity. [ii] Players’ last names are omitted for privacy reasons. [iii] The statistically minded sticklers in the audience are probably complaining that mixed models don't have p-values associated with them, and they're right. The p-values reported are the results of a likelihood ratio test, comparing our full mixed-effect models with reduced models without the fixed effect of the bat type. More details are available on page 11 here.
[iv] Roughly 20 percent of the 3,200+ balls tracked were classified as “Hard Hit”
Jeff Long is an author of Baseball Prospectus. Follow @JeffLongBP
|
Playoff Prospectus: PECOTA Odds and ALCS Gam...
Free Agent Watch: Week Seven
Nice work, guys. Regarding the claim of "more efficient power transfer", there are two ways to interpret that. One if more efficient power transfer from bat to ball. Two is more efficient power transfer from batter to bat. I suspect they mean the latter, which is probably similar to the claim of higher bat speed. I can think of no scientific reason why the first interpretation would be the case,and it would be very easy to show that in a laboratory experiment at Washington State.
Why not hit off a tee, thereby removing one extra degree of variability. I guess I am skeptical of the claim of higher bat speed except perhaps as a long term effect due to greater comfort, less chance of hamate injury, etc.
I am curious about your looking separately at exit speed and distance. With HitTrax, distance is a calculated quantity based on exit speed (and launch angle).
Once again, very nice experiment and analysis.
Those are all good points, and the short answer is I agree with you.
The long answer is:
1. I also suspect they mean the latter (more efficient transfer from batter to bat). Because the ball will always hit the same part of the bat, I know they do some funky things with the inner walls of the metal bats (which, incidentally, is why we used wood bats in our study). But if those tweaks increased the collision efficiency, which I don't have to tell you is carefully controlled.
2. I think you might be onto something with the "long-term effect" explanation. Remember, these results were done *after* a two-month adjustment period with the Axe Bats. So it's not unreasonable to think that what we're seeing is essentially related to the improved ergonomics.
3. You raise a good point about distance being directly dependent on exit velocity. On the other hand, distance is dependent on exit velocity in the real world, too, and at least with the HitTrax system you don't have to worry about atmospheric conditions and the like. Maybe the solution is just not to use distance at all in future studies like this, because they're too closely related?
Thanks!
Bryan...regarding you 3rd point, I am suggesting that you not use distance, since it is sort of redundant with exit speed. That is, it's not really new information. And, as I said, it is not actually measured with HitTrax.
Do you know how many MLB players are using this bat? If I recall, Dustin Pedroia.
I am very fond of doing experiments under controlled conditions (having spent a career doing such things in my own field). This was a very nice study.
There's a list of Axe Bat's pros in the bottom of Jeff's earlier article (http://www.baseballprospectus.com/article.php?articleid=27999#_edn2). There are a lot of Red Sox, which to me sounds like Pedroia stumped pretty hard for it.