In January, as Northwestern football players entered the transfer portal and exited to greener pastures, friends and I were texting: "What does this say about the health of the program? Are exiting transfers a useful metric for program quality? If things are changing, what were they before?" We discussed several measures of quality, but found those standards lacking.
The program facility has pedestals for division, conference, and bowl trophies; we are told to "Expect Victory." One friend suggested a five-win floor as an expectation, with anything less a disappointment. All these ideas point to an underlying question: What kind of program is Northwestern Football?
The most obvious place to look for program effectiveness is win-loss record. It's easy to find and provides the six-win threshold for bowl eligibility. However, seasonal differences in scheduling and the relative strengths of conferences mean that season wins lack predictive power. The gold standard for predictive statistics in college football is Bill Connelly's SP+, a "tempo- and opponent-adjusted measure of college football efficiency" which focuses on a team's likelihood of performing well on future downs. The higher the SP+ rating, the better the team is: bad teams get negative ratings, average teams hover around zero, and good teams are positive.
To get more from SP+, I've done a casual examination of Northwestern football with limited data, reflecting my own efforts to work through the wild swings in outcomes from 2017-2021.
FBS SP+ in the Fitz era
I focused on SP+ across the 130 Football Bowl Subdivision teams while Pat Fitzgerald was the head coach at Northwestern, the 2006-2021 seasons. Fitz's tenure also represents very steady occupancy of head-coach-adjacent positions, with Athletics Director Jim Phillips (2008-2020), Offensive Coordinator Mick McCall (2008-2019), and Defensive Coordinator Mike Hankwitz (2008-2020) joining Fitz to create an era of stable overall football program leadership.
To characterize FBS programs, I used season-ending SP+ ratings retrieved from https://collegefootballdata.com. I calculated the mean and standard deviation of each program over the 2006-2021 seasons (16 seasons total). The mean rating tells us how good the program has been over that time. The standard deviation tells us how consistently the program has performed at that level. The x-axis is average performance, the y-axis is standard deviation. The farther right a dot is, the better the program is. The higher it is, the more inconsistent it is.
Sixteen years of college football, boiled down to two characteristic values.
Before we ask this data what to think about Northwestern, let's do some quick checks and see if the data is consistent with conventional wisdom.
Teams that win a lot should have high average SP+ ratings. From 2006-2021, Alabama (28.96), Ohio State (26.32), Oklahoma (24.04), Georgia (23.16), and LSU (22.88) round out the top 5 in average ratings. These teams won 10 of the last 16 national championships. Teams with high average SP+ performance are rewarded with wins, playoff berths, and championships.
The two dots at the far left are stinkers: UMass (-23.64) and New Mexico State (-22.64). If you asked me to name FBS teams before this exercise, these two would not have made my list (please do not ask me to do this). The Minutemen only joined the FBS in 2011 after competing in the FCS, winning a title in 1998. UMass won two games in their last two normal seasons. New Mexico State, another independent, won two games each in 2019 and 2021 (both teams' 2020 schedules were cut short). Low average SP+ dooms teams to obscure toiling and low-win seasons.
What does consistency look like? Starting at the low consistency end (high standard deviation, top of the chart), we find schools like Western Kentucky (13.75), Florida State (11.53), and Coastal Carolina (12.64). Florida State's record is all over the place, with a national championship in 2013 and a 3-win season in 2020. Coastal Carolina's five-year FBS history has seasons of 3-9 (2017) and a top 15 finish (2020).
Programs with low standard deviations are very consistent; they are what they are year in and year out. Virginia has a middling average rating of 2.06, but rock-solid consistency with a standard deviation of 3.9, the best in the data. Oklahoma is consistently good (SD 4.71, mean 24.04) and UNLV is consistently bad (SD 4.01, mean -15.71). Here among the steadies is where we find Northwestern.
Northwestern's SP+ ratings
Northwestern's average SP+ rating is right in the middle of the pack, just above average at 1.08. More shockingly (to me, at least), Northwestern has the 17th most consistent SP+ rating in the sample with 5.71, right between Iowa and Ohio. Northwestern is mediocre on average, and they are consistently mediocre.
That purple circle is where we've all been living for the past 16 seasons of Northwestern football.
Northwestern's closest statistical neighbors are not anything to celebrate. Maryland is very close, with an SD of 6.15 and mean of 1.25. Colorado (SD 5.51, mean -2.54) is similar, just on the other side of average, as is Houston (SD 5.85, mean 3.99), just a little better.
Also interesting are Northwestern's aspirational rivals which do not compare. Cross-border rival Wisconsin is also very consistent (5.05), as is Nebraska (5.54). Both of these programs are much better than Northwestern, with average ratings of 18.97 and 11.98, respectively. Northwestern has stolen a few wins from each in the Fitz era, but the data tells us that those are flukes rather than something to expect consistently.
Nothwestern's performance over time
The clearest trend in Northwestern's SP+ is a recent one towards swinginess. This is reflected in both the SP+ rating and win totals from 2017 through 2021.
There's also been a general upwards trend in SP+ rating since 2006 (2021 notwithstanding). These two charts very helpfully illustrate the effectiveness of SP+. Looking at wins alone, one might be fooled into thinking that the 2015 and 2017 teams were of similar quality (each won 10 games!), but the 2015 team was either lucky or benefited from weaker opponents.
The 2017 team is the best, statistically, with an SP+ rating over +11, ten wins, a West Division championship, and a bowl win over Kentucky (whose rating was +2.7 that year).
2011 was a more typical season, with an SP+ of +0.6, 6 wins, and loss in the Meineke Car Care Bowl of Texas to Texas A&M. Similarly, the 2014 team achieved a rating of +2.9, 5 wins, and no bowl appearance. The 2015 team outperformed their SP+ significantly, notching 10 wins on a blah rating of +0.2, a performance rewarded with complete destruction at the hands of Tennessee (+19.5) in the Outback Bowl.
The 2021 season stands out starkly. Northwestern hit their lowest SP+ rating in Fitz's tenure at -10.9; the previous worst was -5.5 in another three-win season, 2019. More troubling than being simply disappointing, this -10.9 performance is 2.1 standard deviations from the program's mean performance since 2006. This negative outperformance is a troubling portent for the next few years of Northwestern Football. Either Mike Bajakian and Jim O'Neil will gel better with Fitz and the players, we'll have more coaching changes, or we're looking at several years of 3-5 win seasons.
What this means
Northwestern Football's performance during Fitz's tenure has been consistently mediocre. This is certainly a big step up from the dark ages of consistently bad but might come as a disappointment to fans plugged into the athletic department's aspirational messaging or relishing too dearly marquee wins over historically great programs (I personally solidified my Northwestern fandom as a freshman at the 2004 Ohio State game, so... guilty).
The athletic department and team leadership will probably keep holding up The Three Trophies (West, B1G, Bowl) as a seasonal aspiration. That's good. They should. Northwestern Football certainly can be good enough to compete for those. We as fans should remember that those are motivating goals for the athletes, coaches, support staff, and recruits who can actually make them happen. While they work on that, we can hold back our shock and disappointment when great players transfer from our mediocre program to great ones. We can celebrate stolen wins and drown our sorrows when we fail to score second half points against bad MAC teams. Those kinds of things can be expected from mediocre programs like ours.
With the recent changes at offensive and defensive coordinator, we're probably looking at a new era of Northwestern football in any case. With baseline performance from 2006-2019 established, fans also have a new method to compare the team's total quality to the prior regime, something more predictive than noisy measures like wins and transfer portal activity.