Northwestern football, "luck," and the 8-2 record that perhaps shouldn't be

Stats can be deceptive. Stats can be cruelly misleading. But they can also be extremely useful, and stats — and very good ones at that — say that Northwestern is not one of the 50 best college football teams in the country.

After an ugly 21-14 win over a bad Purdue team — and with teams like Duke and Stanford not looking as strong as originally thought — the Wildcats fell 11 spots in Bill Connelly's S&P+ rankings, all the way to No. 58. They come in behind teams such as Washington (4-6, No. 25), Louisiana Tech (7-3, No. 35), Minnesota (4-6, No. 41), Illinois (5-5, No. 44), Utah State (5-5, No. 47), Nebraska (5-6, No. 54), Virginia Tech (5-5, No. 55) and Western Michigan (6-4, No. 58).

So it's time to face the question that has been picking up steam over the past few weeks: Is Northwestern "lucky" to be 8-2?

Within this discussion, there needs to be a distinction between "successful" and "good." Teams can play well and not have success. Teams that are decidedly average can have a ton of success.

Northwestern has been remarkably successful if success is only measured by wins and losses — which it is. With regards to what has already happened this season, without considering the future, win-loss success is all that matters. It does not matter how Northwestern has reached 8-2, it only matters that it has.

But the true effectiveness of a team should not be measured in wins and losses. There are far better indicators of that. There are measures that break down those 10 games or data points — an extremely small sample size — into over a thousand data points for any given team. S&P+ considers teams in terms of every single non-"garbage time" play that they participate in throughout the season. It takes all three phases of the game into account and adjusts for opponent. When upcoming games are in question, that true statistical ability is what matters. Past win-loss success no longer does. Metrics like S&P+ are far more effective at predicting future success.

It's in that light that we begin our discussion of "luck," and it is an important context to understand, because Northwestern has indeed been fortunate this season. It has an S&P+ second-order wins differential of -2.6 (explained later), the lowest such mark in the country. Essentially, factoring in every non-"garbage time" play of this season, Northwestern has played well enough to win, on average, if the first 10 games of the season were simulated over and over again, just 5.4 games. Based on success, NU is a top 20 team in the country, but based on how well it has played, it is not.

These stats should not diminish the accomplishment that an 8-2 record is. It does not take away reason to celebrate close wins over Penn State and Nebraska. What it does do is explain why Northwestern is and should be a double-digit underdog at Wisconsin this weekend. The Wildcats simply haven't been as good as their 8-2 record indicates.

What we're talking about here isn't luck in the literal sense. It is not external forces acting in Northwestern's favor. It is not things that are out of the Wildcats' control simply falling into place for them.

What we're actually talking about is random variation. Here's how the Wildcats have been lucky: They've essentially (and inadvertently, by chance) allocated as much bad performance as possible into two games (at Michigan, Iowa), and into six others (excluding Minnesota and Eastern Illinois) they've allocated just enough good performance to win all six.

Now, that's not a conscious decision to do that. Northwestern is trying to be as good as it can be on every single play, "to go 1-0 each week," as head coach Pat Fitzgerald so often recites. If it were a conscious decision, every team would be trying to do exactly what Northwestern has done. Instead, it's random variation. So what we're talking about isn't "luck." It's good fortune when it comes to random variation.

One of the statistics S&P+ generates is win expectancy — the likelihood, based on all the individual plays that a team ran in a given game, that one team will win. As you can see, Northwestern has played two games that it would win every time, and two games that it would lose every time. It has then played six games with win expectancies between 31 and 71 percent:

Win expectancies can then be used to calculate a season win expectancy, or second-order wins total. The second-order wins differential is simply the difference between a team's second-order wins total and its actual win total. Northwestern's differential of -2.6 is 0.4 points lower than any other team's in the country.

In past years, the Wildcats have been on the other end of the spectrum. In 2013, to a lesser extent, they allocated a just barely insufficient amount of good performance into five games (Ohio State, Minnesota, at Iowa, at Nebraska, Michigan). Northwestern didn't have entirely poor fortune that year; it allocated a lot of bad performance to the Michigan State and Wisconsin games, while allocating just enough to the Illinois game. But its second-order wins differential that year of 1.2 showed that the Wildcats had actually played well enough to, more often than not, if the season were repeated over and over again, make a bowl.

The counterargument to the 2015 Wildcats simply being fortunate and the 2013 edition being unfortunate is that there is something qualitative about this team that makes it more likely to win seemingly 50-50 games than the average team.

Or maybe, with a great defense and a putrid offense, Northwestern, a separate argument might go, knows that it has to win games ugly. Its comfortable wins, one might say, turn into close wins on paper because the coaching staff gets conservative with a lead.

But in reality, all but one of those six close games have been come-from-behind wins, and in the one that wasn't, the game was tied with five minutes remaining in the fourth quarter. Northwestern has also outscored its opponents in those six games 33-25 in the fourth quarter.

Is it possible that there is something about Northwestern's 2015 team that makes it more likely to win close games than lose them? It's possible. Perhaps it's even probable. If S&P+'s win expectancy says Northwestern played well enough to win a given game 50 percent of the time, there's a decent chance that Northwestern, because of other unquantifiable variables, actually had a greater than 50 percent chance to win the game.

But just because Northwestern has won all its close games does not mean that that percentage should be 100 percent. It probably shouldn't even be 60 percent. Six games is a remarkably small sample size, and there's very little a team can do over the span of a football season to prove that those variables are present and working in its favor. Thus, we should trust the years and years of previous examples, and the stats that arise from them, not six games.

Furthermore, even if Northwestern did have something about it that would make a statistically-calculated 50 percent win expectancy for almost any other team a 60 percent win expectancy for NU, it would still be considered fortunate. If in its six non-zero percent and non-100 percent games, Northwestern's win expectancies were 10 percentage points higher than the statistical formula would suggest, its second-order win differential would still be -2.0. That would still be the third-"most fortunate" mark in the nation.

The discrepancy in wins and expected wins is easy to see through concrete examples too. For instance, what if Solomon Vault's two kick return touchdowns don't happen against Duke and Penn State? What if they happen against Minnesota and Iowa instead? Northwestern is no less of a team, but it is likely 6-4, which is far closer to its season win expectancy, and far closer to being representative of its true ability.

But then again, good special teams scheming may have exploited specific breakdowns in Duke's and Penn State's kick coverage. Statistics, though, can't know that.

So, are S&P+ or any of these other stats gospel? Are they absolute? No, of course not. For example, Northwestern is ranked behind Illinois and Minnesota, but you're not going to see any of our writers argue that the Illini and Golden Gophers are better teams than NU. At least I hope not.

But often times skeptics want to discredit statistics because they aren't perfect. And that's where they err. As much as we want them to be, statistics aren't perfect evaluators, nor are they all-encompassing, and they aren't going to be anytime soon. Any metric that tries to quantify any sport has its imperfections. Instead, what we should be in search of are the stats that are the most perfect — the ones that have the fewest imperfections. When trying to judge a team's true ability, the imperfections of systems like S&P+ and F/+ are minimal compared to the imperfections of win-loss record.

Northwestern is thus fortunate to be 8-2. The Wildcats might not be as bad as that No. 58 S&P+ ranking suggests, but they're also probably not a top-40 team. They're almost certainly not a top-30 team. That doesn't delegitimize the eight wins, nor does it make Northwestern's season any less impressive. But NU's good fortune and its sterling record in close games are likely to regress to the mean over time. All of this therefore does signal that the Wildcats are far less likely to win their remaining two games against Wisconsin and Illinois than you might think.