116: Résumé metrics
A comparison of résumé metrics ahead of the first college football rankings tomorrow night.
The first college football playoff rankings debut tomorrow. The format of the playoff is tripling in size, including 12 total teams after a four team format over the past 10 seasons.
Rich Clark, the executive director for the playoff committee, shared some insight last week into how the committee with assess certain teams.
These quotes from Clark stand out:
"Record matters, but we're not trying to pick the most deserving teams, we're trying to pick the best teams. They're going to consider record, of course, but they're going to look at strength of schedule. They're going to look at head-to-head competition, how teams perform against each other. If a team rolls through a schedule that's a very easy schedule, it's kind of hard to judge them against a team that lost two games but has a really tough schedule."
Clark is right. It’s hard to judge teams that play different schedules.
There are smart people that can help with that judgement. The majority of the data is even public.
For this exercise, we’re going to compare ESPN’s FPI and strength of record metrics with Brian Fremeau’s FEI ratings and strength of record or résumé metrics1.
The table below shows the 24 teams with at least a 10 percent chance or better to make the playoff according to ESPN. The teams are grouped by conference and any common opponents among the 24 teams are also shown.
A few takeaways from the chart . . .
Scheduling and mega-conferences
One of the biggest challenges this season is that several of these teams from the same conference don’t even play each other. It’s going to be hard for the committee to look at a lot of head-to-head competition even within a league. For example, the SEC and Big Ten represent 11 of the 24 listed teams, but none of the seven teams from the SEC play the four teams from the Big Ten this regular season.
There were only four non-conference games total between the two power leagues. The SEC won three of those four games, none of those matchups included the four teams from the Big Ten in this chart.
Strength of schedule ≠ strength of record
Chris Fallica points out that Indiana could finish the season 11-1 with zero wins over a team receiving a single point in the AP Poll. The schedule hasn’t been difficult, but the Hoosiers still have dominated teams to the tune of a +160 point differential in Big Ten play. At least, Indiana plays Ohio State at the end of this regular season.
Miami is the only team on this chart without a common opponent amongst the other 23 listed teams. Even with that absence, the Hurricanes have the third best résumé in both systems.
Seeding rules
The rules of the playoff aren’t an exact science. The four highest-ranked conference champions will be seeded one through four and will receive a first-round bye. The fifth conference champion will be seeded where it was ranked or at No. 12 if it is outside the top 12 rankings.
This is where Boise State could cause some problems because the Broncos own a higher FEI rating percentile than all four Big 12 teams on the chart as of today. If the Big 12 continues its chaos, and BYU were to lose a regular season game or the Big 12 conference championship, Boise State could be in-line for a first-round bye and top-four seed.
Conference championship games helpful or hurtful?
While a conference championship game could hurt a team like BYU or Boise State, could the lack of a game help Notre Dame or Washington State?
The Irish and Cougars still likely need to run the table to feel safe. Notre Dame plays Army in Yankee Stadium on November 23, which proves to be a large game for its résumé. Washington State has no marquee matchups left playing Utah State, New Mexico, Oregon State, and Wyoming.
The lack of divisions and tiebreakers could keep a good team like Indiana or Georgia out of a conference championship game too, but if these teams are safely in the playoff field, would their coaches actually prefer the extra rest?
Snapshots
The committee will share its first snapshot tomorrow night. It should reveal more about how they’re looking to pick the best teams.
Similar to college basketball, adopting a wins-above-bubble metric could help provide clarity to the committee's decisions. I imagine that’s more difficult to define in football with smaller sample sizes, but again, smart people and systems are attempting to define record strengths today.
While not featured in the chart2, Bill Connelly’s Résumé SP+, is another useful tool. It’s defined by how the average SP+ top-five team would be projected to perform against a team’s schedule in terms of scoring margin. It adds a seven-point penalty for every loss a team has suffered.
For what it’s worth, Indiana and Miami are first and second in Résumé SP+ as of today. In my opinion, this is a good indication of a résumé metric because you should not be penalized for a win and awarded for a loss.
Scraps
Last week I put together a progression chart using FEI ratings, and did not include in the newsletter, so adding an updated version below.
This arbitrarily picks four teams that have exceeded preseason expectations and four teams that have disappointed. The chart tells the story, and holy cow ‘Noles.
🏀 College Hoops is back
As I’m writing this newsletter, there are tons of women’s and men’s college basketball games taking place right now. November 4 is the first official day of games and it’s no where near what the old tip-off marathon used to feel like seasons ago.
Anyhow, here are three things I’m curious about this college hoops season:
Does minutes continuity matter as much as it once did? How do we measure experience and transfers within the same leagues or with new/former teammates or the same coach?
If WAB (wins-above-bubble) debuts on Team Sheets this season and the metric is derived from NET, will we finally understand the raw rating of the NET? And will quadrants become less of a nuisance? Remember WABStick?
This one is selfish. Given North Carolina’s personnel, will the team’s pace and space style be as effective as the old shot volume style of the past?
March is a long way away, I’m just trying to get through today.
Thanks for reading this far, it was a long post. As far as a recommendation, I just started reading Dean Oliver’s new book, Basketball Beyond Paper. I’m only six chapters in, and it’s very good.
It’s sort of a sequel or extension of his old book, Basketball on Paper, that made the four factors more mainstream. I remember reading the first book long ago, and even created my own four factors calculator.
Please subscribe if you so choose and check out this app that continues to surface more college football data as we’re in the final month of the regular season.
SOR stands for strength of record. ESPN defines this as reflecting the chance that an average Top 25 team would have the team's record or better, given the team’s schedule. Unfortunately, there is no raw rating next to this rank, so it’s a bit murky on the distance between teams. It’s also unclear how a Top 25 team is defined?
FEI ratings are using the GWD (Good Win Differential?) résumé ratings from bcftoys.com. GWD uses a strength of schedule rating that represents the number of losses a team two standard deviations above average would expect to have against the schedule of opponents, and then shows the difference between a team's schedule strength ratings and its actual losses.
Fremeau also shows EWD (Elite Win Differential?) and AWD (Average Win Differential?). To be honest, the GWD felt like the clearest comparison to whatever the Top 25 metric that is used in SOR.
If you have other ideas, please let me know.
Connelly only shares the top-15 teams in Résumé SP+, so I excluded it in the overall chart. It’s still useful and it lives behind the ESPN+ paywall.