Receipts
We rewound 2025 and ran the model as of three checkpoints, using only what was known then, then checked it against the field the committee actually picked. How it works
- Brier · Week 13
- 0.029
- 2025 playoff odds, lower is better
- Playoff field
- 89%
- Committee model, hidden seasons
- 70% favorites won
- 75%
- 1,990 games since 2014
- Leans agree
- 83%
- Résumé vs simulations
Two-Deep scorecard
67%
of the starters we called before kickoff led their group in that game's box score. 14,835 of 22,002 calls, Weeks 2–12.
Called = the official depth chart when the school published one, otherwise who'd been playing through the game before. The O-line isn't scored: there are no public snap counts to check it against. One game is a noisy referee, especially for running backs who split carries.
How the Two-Deep works →- Quarterback85%
- Running back77%
- Receivers70%
- Tight end59%
- D-line56%
- Linebackers74%
- Secondary64%
Selected setting
2025 backtest
| Season | As of | Brier | Log loss |
|---|---|---|---|
| 2025 | Week 5 | 0.0438 | 0.1489 |
| 2025 | Week 9 | 0.0369 | 0.1189 |
| 2025 | Week 13 | 0.0286 | 0.0940 |
Starting point regressed 25% to average · fades to 20% by game 8 · team shock 6 pts · committee noise 2
Brier is the average squared gap between the playoff odds we gave and what happened, so lower is better. Giving every team the same 1-in-11 chance would score about 0.079. From Week 5 to Week 13 the score went from 0.044 to 0.029, 64% better than that baseline by the last checkpoint. Log loss punishes confident misses harder: a team we called a near-lock that missed costs far more than a coin flip that went the wrong way.
All three checkpoints
Calibration
Predicted playoff chance (across) vs how often those teams made it (up), in 10-point groups. Dots on the line mean the odds meant what they said. Bigger dots, more teams.
Most teams sit in the lowest group, near 0%, because most teams were never in the race, and those almost never made it. The groups that matter are the middle and the top. A dot below the line means we were too confident; above it means we undersold those teams. At the top, teams we had above 90% made it 5 of 7 times, so the high end runs a little hot.
Every FBS-vs-FBS game, 2014–2025
Game predictions
For each of 9,237 games we rebuilt the ratings from games played before that day only, then checked the pregame win chance against what happened.
How often 70% favorites actually won
75%
1,990 games where we had the favorite between 70% and 80% (average 75%).
Home-team win chance
Calibration
Predicted (across) vs how often it happened (up), in 10-point groups. Bigger dots, more games.
Us vs the closing market
| Brier | Log loss | Margin miss | |
|---|---|---|---|
| CFB Index | 0.1833 | 0.5410 | 13.5 |
| Closing market | 0.1608 | 0.4852 | 12.1 |
Same 1,463 games, the only ones with a historical closing line on file so far. The market wins, and it should: it sees injuries, weather and money we don't. It's a benchmark, never an input here.
What we changed
- Kept · sigma + home field · held-out Brier 0.1870 → 0.1865
- Kept · tail shrink · held-out Brier 0.1865 → 0.1863
- Dropped · early-season extra SD · held-out Brier 0.1863 → 0.1862
Now: spread 13 pts · home field 2 pts · 1% pulled toward a coin flip. All games: Brier 0.1881 before, 0.1873 after.
| Season | Games | Brier | Log loss | Margin miss |
|---|---|---|---|---|
| 2014 | 763 | 0.1993 | 0.5798 | 14.1 |
| 2015 | 766 | 0.1704 | 0.5117 | 13.5 |
| 2016 | 763 | 0.1894 | 0.5560 | 13.7 |
| 2017 | 787 | 0.1818 | 0.5374 | 13.2 |
| 2018 | 780 | 0.1780 | 0.5283 | 13.5 |
| 2019 | 775 | 0.1740 | 0.5202 | 13.3 |
| 2020 | 650 | 0.2163 | 0.6309 | 12.7 |
| 2021 | 774 | 0.1914 | 0.5657 | 13.7 |
| 2022 | 780 | 0.1988 | 0.5806 | 12.9 |
| 2023 | 792 | 0.1809 | 0.5388 | 13.1 |
| 2024 | 799 | 0.1914 | 0.5600 | 13.0 |
| 2025 | 808 | 0.1805 | 0.5348 | 12.8 |
2014 has no 2013 season on file to start from, so every team opens at average; it's scored but left out of tuning. 2020 was short and strange.
| Games | Brier | Margin miss | |
|---|---|---|---|
| By week | |||
| Week 10 on | 3,817 | 0.1885 | 12.9 |
| Weeks 1–4 | 2,262 | 0.1804 | 14.5 |
| Weeks 5–9 | 3,158 | 0.1907 | 13.0 |
| By matchup | |||
| G6-G6 | 3,875 | 0.1940 | 12.9 |
| P4-G6 | 1,280 | 0.1463 | 15.8 |
| P4-P4 | 4,082 | 0.1938 | 12.9 |
| By size of favorite | |||
| 90%+ | 993 | 0.0595 | 14.5 |
| 50–60% | 2,432 | 0.2466 | 12.8 |
| 60–70% | 2,208 | 0.2270 | 13.2 |
| 70–80% | 1,990 | 0.1844 | 13.1 |
| 80–90% | 1,614 | 0.1257 | 13.7 |
Each season predicted without seeing it
Committee model
We hid one season at a time, taught the model on the other eleven, then asked it for that season's Selection Day top 25. Field = share of the playoff field it got right. Order = how closely the top 25 lined up (1.00 is perfect). Miss = average spots off.
| Field | Top 12 | Order | Miss 12 | Miss 25 | |
|---|---|---|---|---|---|
| Committee model | 89% | 92% | 0.95 | 1.3 | 2.0 |
| AP poll | 94% | 97% | 0.97 | 0.7 | 1.7 |
| Record strength alone | 89% | 83% | 0.84 | 2.3 | 3.7 |
| Our old committee model | 78% | 84% | 0.83 | 2.6 | 4.5 |
| Losses, then rating | 68% | 66% | 0.75 | 4.9 | 7.2 |
Averages over 2014–2025. The AP poll at Selection Day comes out after a season of watching the committee, so it's a tough bar, and it beats us on most of these. Week to week, starting from the last real ranking, the model misses by 2.1 spots on average vs 2.7 for "nothing changes".
| Field | Top 12 | Order | Miss 12 | Miss 25 | |
|---|---|---|---|---|---|
| 2014 | 75% | 92% | 0.97 | 1.4 | 2.0 |
| 2015 | 100% | 92% | 0.94 | 0.8 | 2.4 |
| 2016 | 100% | 92% | 0.94 | 1.3 | 3.0 |
| 2017 | 75% | 83% | 0.93 | 1.8 | 2.3 |
| 2018 | 75% | 100% | 0.98 | 0.8 | 0.9 |
| 2019 | 100% | 92% | 0.97 | 1.5 | 1.5 |
| 2020 | 75% | 75% | 0.88 | 3.2 | 3.8 |
| 2021 | 100% | 100% | 0.97 | 0.8 | 1.8 |
| 2022 | 100% | 92% | 0.97 | 0.9 | 1.4 |
| 2023 | 75% | 100% | 0.94 | 0.3 | 1.5 |
| 2024 | 100% | 92% | 0.97 | 1.5 | 2.2 |
| 2025 | 92% | 100% | 0.95 | 1.0 | 1.6 |
2026
Projected vs committee
Week 10 · 6 exact · 11/12 in the top 12 · 2.4 spots off
Missed: we had ···; they had ···.
Week 11 · 0 exact · 12/12 in the top 12 · 2.5 spots off
Week 12 · 11 exact · 12/12 in the top 12 · 1.3 spots off
Résumé vs simulations
Schedule leans
83%
Where the paired simulations clearly pick a side, how often the résumé reason points the same way. 439 game-and-team pairs from the latest run.
| Résumé effect | Pairs | Same way |
|---|---|---|
| Under 0.1 spot | 100 | 62% |
| 0.1 to 0.5 spot | 130 | 73% |
| Half a spot or more | 209 | 99% |
They split more than we'd like. The résumé check only sees how the committee would rank you; the simulations also play out conference races and auto bids, so they win ties. When the simulations clearly disagree, Schedule shows their side or calls it a wash.
Our top 25 vs the committee
The committee has released its 2026 rankings. The side-by-side lands here with the next update.