CONCEPT
Benchmark Saturation
The point at which frontier models reach the ceiling of a benchmark — 95%, 98%, 99% — after which the benchmark no longer distinguishes between systems and becomes useless as a progress measure.
Benchmark saturation is the condition in which a benchmark's best reported scores approach its theoretical maximum closely enough that further improvements are indistinguishable from noise, annotator disagreement, or test-set labeling errors. In 2024–2025 the list of saturated benchmarks grew quickly to include GLUE, SuperGLUE, MMLU, HumanEval, GSM8K, HellaSwag, ARC-Challenge, and many others that were once the frontier of difficulty. A saturated benchmark retains historical value as a sanity check — any new model must still solve it — but loses its power to discriminate between the capabilities of current frontier systems. The AI field is currently in a sustained sprint to produce new, unsaturated benchmarks faster than the old ones fall, and losing.
In The You On AI Field Guide
The saturation curve for a typical benchmark has become predictable. A benchmark is released with a state-of-the-art around 50–60%. Within eighteen months, a frontier model passes 80%. Within three years, the leading score is above 90% and the gap between top systems