CONCEPT
The 9.7 Percent
The probability superforecasters assigned to AI benchmark achievements that actually occurred in 2025 — Tetlock's paradigm case of radical miscalibration even among the world's best predictors.
In September 2025, a follow-up to the Existential Risk Persuasion Tournament revealed that superforecasters had assigned an average 9.7-percent probability to the levels of AI capability that models actually achieved across four benchmarks (coding, graduate-level science, vision-and-language reasoning, agentic tool use). Domain experts performed better at 24.6 percent but were still off by a factor of four. The finding was humbling to both communities: the people who understood AI best and the people who were best at prediction had both systematically underestimated the pace of progress. The 9.7 percent became a symbol of irreducible uncertainty in the face of rapidly accelerating, genuinely novel phenomena — a paperweight sitting on every confident claim about AI timelines.
In The You On AI Field Guide
The underestimation was not random error but systematic. Both superforecasters and AI experts were using reference classes drawn from pre-2022 AI progress, when capabilities improved gradually and benchmarks were beaten incrementally over years. The 2022–2025 period broke those reference classes: scaling laws encountered a