Do Stars Matter?

In the past decade, publications like 247 Sports and Rivals have brought the world of recruiting right to the fans, and it’s created a monster. Never before have fans or the media had such an inside look into the top players in high school football, and what the experts think of them. But how much do these star rankings actually matter? Are they really a good measure of anything significant? And if they are, how are the Hokies stacking up? That’s what I’m going to dive into.

It’s no secret that the Hokies have not been recruiting at an elite pace. Over the past decade, matchups against college football’s elite showed the Hokies to be outmatched in skill and physicality. That’s not to say the Hokies weren’t competitive in games against Clemson, Notre Dame, Ohio State, Miami, etc., but there were moments the Hokies were simply outmatched. It seems like the space between college football’s elite programs and Virginia Tech is starting to expand. But is this due to recruiting?

To answer this, I’m going to compare Virginia Tech’s performance on the field and on the recruiting trail to those of other Power 5 programs, with a more concentrated look at the teams that make the New Year’s Six bowl games. That being said, all data is from the 2014-15 season to 2019-20. Recruiting ratings are gathered from the 247 Sports Composite, and include data from all Power 5 teams as well as Group of 5 teams that finished the season in the top 25.

I’ll be using a 3 year average to assign recruiting ratings. For example, the 2014 Virginia Tech recruiting rating is the average of all player’s individual ratings from 2011-13. This gives me the best look into the player’s currently on the roster that will most likely be contributing.

Also, reader discretion that this is about to get really nerdy and mathematical. I’ll do my best to explain everything I’m doing.

But to start, let's talk about heteroscedastic versus homoscedastic variables. Heteroscedastic variables are stats where different values can have different errors, or factors, associated with them. Homoscedastic means that there is just one error contributing to the value that is standard throughout all values. In football, there is only one stat that directly contributes to winning the game, points. The only thing that goes into the final score of a game is how many points both teams score. You don’t get points for turnovers, passing yards, etc. Nothing else helps you directly win the game.

Recruiting is heteroscedastic, factors are different for each individual team. How are the teams you are competing against recruiting? How are you developing these recruits? Do you have the same access to these recruits as another school? Because of this, we cannot answer our original question, or null hypothesis, of whether or not recruiting better creates more wins. Recruiting rankings don’t contribute to the final score, so we can’t say that it causes winning. (That’s the old correlation doesn't equal causation effect.)

However, we can ask a new question: How much does recruiting impact on the field success? That’s what we’re going to measure. But first, let’s start at the top of college football and see how Tech is sizing up to successful programs around the country. I think it's important that we get a macro understanding of the national recruiting landscape so we can better understand where Tech falls in all of this.

Since its inception in 2014, Tech has not made it to a New Year’s Six bowl game. The lack of a big-time bowl game appearance has been a signal of Tech slipping off the national pedestal in the last 8-9 years. Let’s compare Tech’s recruiting numbers to those of teams in New Year’s Six bowls:

What this shows us is that the roster makeup of the Hokies has been in the lower part of the teams competing at the top of college football in 3-year average recruit ratings. When you exclude the Group of 5 teams (in grey), the Hokies are really on the fringe. Unfortunately for Tech, the gap has increased.

The increasing gap between college football's elite and the rest of the teams on the recruiting trail has been looked at as one of the main factors in teams like Alabama, Clemson, and Ohio State have been consistently dominant. But is that true?

Next, we're going to look at every Power 5 and/or top 25 team's winning percentage and 3-year average recruiting ratings from 2014-today. This is a lot of data, but it's important that we take as macro of an approach to this as possible. This is because proving that winning teams have high recruiting ratings doesn't tell the whole story. We need to understand how recruiting ratings impact both winning and losing teams. If it's to be believed that recruiting ratings impact winning, then both sides of the story need to be true. Here is what the data says:

This is a bivariate regression model in JMP. This type of model can make sense of huge amounts of data like what we're working with. It tells us a lot of key information like the mean winning percentage and the line of best fit, or expected value, for all the values in the plot. The dots that are highlighted in black are Tech's values. So, you can see that Tech has been hovering around the mean winning percentage and expected value over the past 6 seasons. This tells us that Tech has been performing roughly where they are supposed to be in terms of winning and recruiting. But are the two actually related?

You may have expected to see a graph with values arranging from the bottom left to the top right. This would be a data set with a high R-square value, which measures how much of the data fits within an expected error range. However, this graph looks nothing like that, more like a smattering of data points with no order. In fact, the R-square value for this data set is relatively low at around 52%. While this may be seen as an argument against the hypothesis that recruiting ratings impact winning, it actually makes sense if you know college football.

The low R-Square value is due to teams such as Navy, Boise State, Cincinnati, UCF, and...oh yeah...Wisconsin. These are teams that recruit at a lower level than they win, whether it be due to level of competition or offensive schematics. They are located in the upper left of our graph, way outside of the expected value range that the R-square would measure. The inverse can be said for teams such as Tennessee, Texas A&M, Miami, and Florida State. Those teams recruit at a higher level than they win and are located in the middle and bottom right of the graph.

The more important stat from this model is the p-value, which measures how significant a variable is to the overall model. Simply put, the p-value is the probability that any additional values added to the data set would go against what the model is currently predicting. So, a low p-value means that the variable, which in our case is recruiting ratings, is significant in determining the outcome, winning. For most models, a variable is considered significant if an additional value would have a 95% chance of falling within a normal distribution (a p-value of 0.05.)

This model has a p-value of less than 0.0001.

Recruiting ratings may not directly cause winning, but they are as mathematically significant in the matter as you can get.

For years, Tech fans have used the example of Wisconsin to show that a program can compete on a national scale without elite recruits. That argument fails to realize that Wisconsin has historically beaten up on the weaker half of the Big Ten conference, and has lost 8 straight to Ohio State since 2011, including 59 and 31 point losses. Wisconsin has ridden their success in the weaker half of the Big Ten to two New Year's Six bowl appearances since 2014, where they're 2-1. The lone loss was last year to a very talented Oregon team. The two wins? Western Michigan and an overinflated ACC Coastal champion in Miami.

I don't mean to drag Wisconsin through the mud. They're an immensely successful program that is certainly playing at a higher level than Tech over the past decade. However, when faced against college football's most talented teams, which Oregon and Ohio State certainly fall under, the Badgers lose. Do Tech fans really want to settle for just making it to the big game?

I bring up this point for my third piece of analysis, which circles back to New Year's Six bowls. Teams with the higher 3-year average recruit rating are 22-14 in New Year's Six bowls. Outside of 2014, which was a wild season, to say the least, higher-rated recruiting teams are 21-10. The following chart shows the 2014 New Year's Six games with the winning teams on the left side. If the recruiting rating is green, then the higher-rated team won (vice-versa for red.)

This season was an anomaly in terms of how the New Year's Six games shaped out. You can just look at those matchups and see how wild it was that Georgia Tech, Mississippi State, and Ole Miss, and Michigan State all made it in. Heck, Ole Miss had to blatantly cheat for 3 years just to get beat by TCU. The years since have been more on brand for college football.

This tells us that not only are recruiting ratings significant in a teams winning percentage over a period of time but also in big game matchups. Although a 22-14 record is by no means a stat line to put your money on, the recent trend of higher rated teams winning is something to think about in regard to the direction of high level college football.

So, to answer our question definitively...yes, stars matter.

Maybe not in the way that fans and media have traditionally argued, but stars matter nonetheless. The outlier teams like Wisconsin and Tennessee are very much unique cases. On the whole, if you want to win in college football on a regular basis you need high level talent. And if you want to win trophies, you have to be nothing short of elite.

For Tech, there is definitely some ground to make up on the recruiting trail. That's not to say that the last 4-5 classes have not had immensely talented players or that the next few classes will not. But with the current trend in college football, it's far more likely that Tech will continue to post mediocre to good records if recruiting remains stagnant. I don't think Tech is far off the pace, however. Teams in weaker conferences like Oregon and Clemson were able to build their program with top 20 classes nationally. For the ACC, we can see that there is a clear window of opportunity to be the most talented team on the field consistently.

We can see that Tech has performed well in the ACC over the past 6 seasons with fairly average recruiting numbers. This points to the staff's ability to get the most out of what they have and develop players. The key for Tech in the future will be starting with better raw materials to create an even better product.

The top of college football is not as far away from Tech as it may seem at times. It's a tough climb, however, and the numbers tell us that it starts on the recruiting trail.