Iowa high school football:
Week 3 is the trickiest week to rank.
It is the first unbiased rankings — but are these new unbiased rankings better than the rankings that could have been published that retained last year’s bias?
This post is a starting point for this analysis. I will revisit this post in a week and compare the predictions of three systems:
- model (1) unbiased rankings
- model (2) “unbiased” rankings, with each team including a 0.00 game strength (generally 60% this year’s data, 40% 0.00 game strengths)
- model (3) biased rankings, with last year’s team strenghts (generally 60% this year’s data, 40% last year’s team strengths)
model (1)
Advantages: completely unbiased (it really “feels” good)
Disadvantages: inaccurate “transitive” effects are more likely to cause inaccuracy; model is overfitted; model does not provide adequate room for randomness
Team A wins by 10 over team B
Team B wins by 10 over team C
Team C wins by 10 over team D
Team D wins by 10 over team E
therefore, team A will win by 40 over team E.
And thus you see the problem.
The effect of this problem is that the teams “spread” out in a inaccurate way. The top team in the rankings will be 200 points better that the worst team. If you look that the historical reports, about 120 points is the typically spread from the best to the worst team.
model (2)
Advantages: largely retains an unbiased nature: no team is treated differently in the model
Disadvantages: will incorrectly dampen the team score of a team that has played a hard schedule; will underestimate the game margins for non-competitive games
Note: This is the model that is currently published this week.
model (3)
Advantages: introduces bias
Disadvantages: introduces bias
Ok, this is delicious! Is it an advantage or disadvantage? Well, the answer will arrive if it predicts games better or worse.
And that is the standard of the game
========================
Follow-up:
Early results (not complete)
model 1: 95/117 = 79.2%
model 2: 94/117 = 78.3%
model 3: 96/117 = 80.0%
All three models are relatively close in picking winners.
I will follow up later and see how close the game-margin predictions were to the actual game results.