Statistics on Offensive Predictability

Editor's Note: Bumped to the front because this is really, really well done. --Joe

Based on data available from cfbstats.com, I took a statistical look at how predictable VT’s offensive play calling has been thus far this season (note that I believe the last game is missing). The site provides basic information on every play in every game, but does not give information on specifics like formation, whether a pass was completed or not, etc. So this analysis (based on logistic regression if you’re a geek) simply looks at the most basic aspect of predictability – whether or not we rush or pass.

When we talk about predictability, the most important aspect is whether, based on easy-to-digest information, you can reasonably predict well what will happen. Predicting a team is 50% likely to rush is not particularly helpful because you have to plan for both. Predicting a team is 80% likely to rush (or 20%, which means 80% likely to pass) is very helpful.

This analysis was done by modeling the likelihood of VT rushing based on down, distance to go, position on field, score difference, and quarter. I consider these to be the major factor an opposing coach would use to generally guess what the offense will do (other than the obvious other factor of what have they done all game).

First, here are the odds of VT rushing on 1^st and 10 based on the score difference (positive differences mean VT leads…negative differences mean they trail):

This general pattern is fairly obvious, as most teams with a lead want to kill clock by rushing and when trailing want to save clock by passing. But there is also an element of what you do best and how important that time is – in other words, if you’re good at rushing then early in the game you should probably continue to rush and not worry about the clock even if you trail, and start worrying about the clock later in the game.

So I grouped the graph by what quarter of the game it is and fit a regression to each quarter:

It appears that even in the first quarter VT is already trying to save or kill clock based on the score. This provides for some very predictable and less-than-optimal play-calling.

Next I compared the original graph to what all teams do to see how we match the norm:

VT appears to be much more likely to change play-calling as a reaction to the score difference rather than call what is most effective. The average team calls for a rush about 10% more often with a 20-point lead than with a 20-point deficit, likely because you don’t start killing clock with the rush until late in the 4^th. VT, on the other hand, calls for the rush almost 60% more with a 20-point lead than a 20-point deficit. Remember, the further you get from the middle (50%) the more predictable you are becoming.

Next I looked at third-down plays. First some more descriptive information:

• On 3^rd and 5 or greater, we have run 3 times out of 45 plays. These occurred when we had leads of 21, 27, and 37 points. While these may be obvious passing downs, you have to at least occasionally surprise the defense.
• Conversely, when we lose yardage on 1^st down (2^nd and >10), we have rushed 7 out of 15 times. This combined with 3^rd downs makes me think we’re trying to catch defenses off guard but are too risk averse about it.
• On 3^rd and 1 or 2, we have rushed all 13 times. On 2^nd and 1 or 2, we have rushed 8 out of 12 times.

Without accounting for the effect of score difference, here are the probabilities of rushing based on field position and yards to go (The red points are where we have data):

So almost regardless of field position, with 3rd and 7 or longer there is less than a 20% chance of a running play. I don’t think teams are stacking the box on us in those situations. But maybe all teams become this predictable in those situations? Well, here's the same graph for the average team:

Or maybe not. Turns out that there are very few areas where the pass becomes as likely as it does for VT, or similarly where it becomes as unlikely...for most teams, even 3rd and 1 from the opponent's 1 still has less than an 80% chance of being a run.
For fun, here is the same graph for VT's 2^nd downs:

And once again, here is the same graph for the average team:

Also, the WR screens aren’t working.