The NFL combine is always a notable event for the sports analytics community. This year the story seems to be Joe Burrow's hand size. Most of the commentary has focused on the theory of why hand size matters versus anecdotal evidence that suggests that it doesn't.
After the size of his mitts came out earlier this week when they measured nine inches at the NFL combine, there was much hand-wringing about whether potential No. 1 overall pick Joe Burrow was still worthy of a top selection.
That’s because there is a belief among some NFL executives and pundits that quarterbacks with small hands struggle to grip the ball, particularly when the weather turns cold and wet later in the season. Among some, there is also a belief that a better grip afforded by larger hands allows a QB to get more revolutions on the ball, according to the Guardian.
But, as an ESPN piece points out, though the size of a quarterback’s hands is a top offseason talking point thanks to the NFL combine, it is a measurement that rarely matters once the season starts.
“While no one is arguing that a strong grip isn’t important for quarterbacks, there is no biological or kinetic proof that hand size correlates in any way to hand strength,” writes ESPN’s David Fleming. “Even with hands the size of catcher’s mitts, Brett Favre still fumbled at an alarming rate (166 times, more than any QB in NFL history and good for 0.55 per game), far greater than that of his replacement, Aaron Rodgers (0.43), whose hands are a quarter-inch smaller.”
This story highlights two key (and underappreciated) challenges in player analytics. In an ideal world, we would have a statistical model that relates hand size to player performance. However, we quickly encounter two problems.
The first is sample size. Or more specifically, the issue is sample size versus the number and complexity of explanatory factors. There are many factors that influence performance of a quarterback. These may be physical, mental, and experienced based. Think about the number of measurements that can be taken at the combine and the data available on collegiate performance.
Additionally, many factors such as height and hand size may be highly correlated. There may also be interactions between variables. For example, maybe it is a combination of strength and hand size that is relevant. The important take-away from this (too brief) discussion is that there are potentially many variables that are related to performance and these variables may effect performance in complex nonlinear ways. The analytical problem is that there are relatively few observations in terms of player performance. How many NFL quarterbacks get significant reps in any season? Maybe 50? The challenge for the analyst is limited observations versus complex and numerous explanatory factors.
The second issue is selection. In the previous paragraph I glossed over the fact that the 50 observations of quarterback performance in any season are produced by a sample of elite athletes that have been given an opportunity to play in the NFL. If we could run our regression of performance versus hand size (and other controls), our results would be for the sub-population of ultra elite athletes that had been selected based on the biases and beliefs of NFL executives. This is an important limitation that should be acknowledged. Analytics are almost always based on very special samples of athletes.
The preceding arguments could be made about the vast majority of "sports analytics" studies. Small sample sizes and selection are perhaps the two biggest reasons why "analytics" support rather than drive player decisions.