The Kurt Warner Effect, or why drafting young QBs may not matter

I.
Age matters. When it comes to drafting offensive skill players, NFL teams tend to use their early draft picks on younger prospects, a pattern that holds up even after controlling for other important pre-draft measurables. Even if coaches and GMs aren't conscious of this trend, it's still there.

Though we still don't know why NFL teams tend to draft younger players (that question is too big for a single post), we can still investigate if a QB's age when he enters the NFL has any impact on his career performance.

From 2000-2013, there were 245 QBs that set foot on an NFL field. First, let's make sure that the trend we found earlier, of younger players being drafted earlier than older players, is true when we only look at QBs. The black dots represent QBs who were drafted, and the red dots at the top of the screen represent undrafted free agents. The green line represents the quadratic line of best fit, and the green shading is the confidence interval.

The Kurt Warner Effect, or why drafting young QBs may not matter

Here are some numbers that will help you win the next time your bar has a trivia night. The average age of our QBs when then entered the NFL was 23.272 years. The youngest QB in our sample was Tommie Maddox, the 25th overall pick in 1992, who was 20.545 years old when he was drafted; the oldest QB was Jeff Garcia, who played in his first NFL game in 1999 with the 49ers at 29.186 years of age, after spending five years in the CFL (the oldest drafted QB was Chris Weinke, at 28.805 years, taken with the 106th selection in 2001).

Here's a histogram summarizing the ages when NFL QBs first joined the league.

The Kurt Warner Effect, or why drafting young QBs may not matter

II.
Of those 245 QBs, 25 had fewer than 10 career pass attempts; 115 had appeared in 16 games or fewer. Some of those QBs had careers cut short by injury, or were rookies who haven't had a chance to throw a lot of passes or appear in many games. We don't want to ignore those players, because it will hurt our power, and because they may reflect the trends we're trying to capture; but we do want to get rid of cases that don't represent what we're trying to measure, like QBs who should really be classified as field goal holders, or human victory cigars. Those QBs don't really reflect what we're trying to measure, and including them might bias our data in unpredictable ways.

The way around this dilemma is to filter the data. In this case, we got rid of all cases that fell below the 25th percentile for both pass attempts and game appearances: specifically, a QB had to have at least six appearances in NFL games and at least 59 pass attempts to make the cut. That whittled down the sample from 245 to 176. Throwing out data is always sad, but to make sure we're actually measuring what we say we're measuring, tough decisions must be made.

Another tough decision is how to deal with the variability in career lengths of the players in our sample. Remember, we're looking at career statistics for all QBs who played in the NFL from 2000-2013, which means there's a wide range in the number of games in which they appeared. Since cumulative statistics like passing yards and touchdowns typically only increase as players accrue more game appearances, we know that game appearances is a covariate, and any model that fails to account for covariate will be biased in favor of players with more appearances. This is bad.

There are a few ways to address this. The first is to filter our data to restrict the range of career lengths, which we already did. The second is to include the covariate in our model, so that we're looking at the unique effect of a prospect's age and other predictors on the measures we care about, independent of the covariate's effects. The third is to use rate statistics (e.g. yards per game, attempts per game), so the effects of the covariate are reduced even further. By combining all three of these approaches, we should be able to see if a QB's age when he enters the NFL has any effect on how his career turns out.

III.
We're going to try to use a QB's age when entering the NFL to predict all how his career will turn out, across several metrics: cumulative statistics included pass attempts, yards, TDs, and INTs, and rate statistics included attempts/game, yards/game, TDs/game, INTs/game, completion%, and yards per attempt.

Effects of a QB's age when entering the NFL on specific career outcomes are summarized in the table below. I'm including any effect with a p-value <.1 for discussion, but let's be really clear: I'm running a lot of tests (ten in total), and I'm not correcting my p-values for multiple comparisons. These results should be interpreted as exploratory analyses that can serve as a starting point for future investigations.

MeasureEffectEstimateUnique error explainedFp
Pass attempts/gameAge26.3551.3%2.793.0965
INT/gameAge29.0202.7%2.919.0894
Completion%Age.2592.1%5.213.0236

The positive quadratic effects of initial age on Attempts/game and INT/game, in the absence of any linear effects, suggest that the QBs with the lowest and highest initial ages had more pass attempts per game, and that those QBs used their extra opportunities to throw more INTs. If anything, it suggests that NFL teams might rush out QBs who are too young, or might think that older QBs are better prepared to handle a heavier workload than they actually are, but due to the tiny effect sizes (important effects have much bigger F's), I don't think it's safe to draw any real conclusions about these quadratic effects.

The only test that survived the traditional (uncorrected) p<.05 test was predicting completion% by a QB's age when first entering the league. The positive estimate suggests that as a QB's age of entry increases, so does his career completion%.

This ends up being pretty intuitive: if an NFL team was willing to take a chance on an older QB with no NFL experience, and if that QB actually stuck around long enough to meet the criteria of our filter, then he was probably pretty polished when he came into the league. Think about guys like Kurt Warner and Jeff Garcia – they were old dudes who played professionally before signing with NFL teams, so they continually accrued meaningful football experience before their NFL debuts.

IV.
The Kurt Warner Effect might be real - it held up against a strong covariate (i.e. games played), and in a followup analysis, it even survived the addition of draft position as a predictor - but I wouldn't bet my house on it. The effect is very small, and as I noted earlier, the significance tests aren't corrected for multiple comparisons. If we wanted to be conservative, we'd need p's<.005 (since we ran ten tests), and none of the age effects come close.

This was just an exercise investigating if a QB's age when entering the NFL might be important. We really can't conclude that from the data, but I don't think we can reject that conclusion either. Not every older QB turns out like Warner or Garcia - there are plenty of Brandon Weedens out there. But if you're an NFL GM concerned about finding a QB who will complete more of his attempts in the long run, it might not hurt to start your search at the older end of the age spectrum.

Questions? Comments? Think I'm an idiot? Hit me up in the comments or on twitter @jimkloet and check out my other football work on rotoviz.

Kurt Warner Image from Getty, all others from Jim Kloet