Noise
Noise: A Flaw in Human Judgment, published in 2021, is the culmination of Kahneman's later work and a coauthored extension of the heuristics-and-biases framework to a different error category. Where the original program focused on bias — systematic departures from ideal judgment in predictable directions — the Noise project focuses on noise — random variability in judgment, such that different judges reach different conclusions from identical cases, or the same judge reaches different conclusions at different times. The book argues that noise is a massive, underappreciated source of error in professional judgment (medicine, law, hiring, forecasting) and that AI systems, by applying consistent algorithms to every case, can dramatically reduce it. But noise reduction comes at a cost: the variability that produces errors also produces innovation, creativity, and the occasional brilliant deviation. A world with less noise is more consistent and less exceptional.
In the AI Story
The book's central empirical claim is that noise — random variability — is as large a source of error in professional judgment as bias, and often larger. Two radiologists reading the same scan reach different diagnoses. Two judges sentencing identical cases impose different sentences. Two hiring managers evaluating the same candidate reach different conclusions. The variability is not merely skill difference; it is noise, and it produces arbitrary outcomes that affect real people.
The AI implications cut in both directions. On one hand, AI systems reduce noise dramatically: the same algorithm applied to the same input produces the same output (modulo sampling temperature). This is genuine improvement, and the medical diagnosis that is consistently adequate is often preferable to the diagnosis that is occasionally brilliant and occasionally terrible. On the other hand, noise reduction compresses the distribution of outcomes toward the mean, reducing the frequency of both catastrophic failures and brilliant successes. Creative breakthroughs are, by definition, outliers.
The Orange Pill's discussion of breadth versus depth can be read through this lens. AI makes breadth cheap — competent performance across a wide range of tasks becomes universally available — but the competent performance is also consistent in a way that human performance is not. The junior developer using Claude Code produces more consistent code than the senior developer's median work, even if the senior developer's best work exceeds what Claude can produce. The distribution compresses; the mean rises; the tails shrink.
The book's argument has direct bearing on the question of whether AI is amplifier or replacement. If AI primarily reduces noise without changing the signal, it functions as a consistency filter — improving average performance but potentially at the cost of exceptional performance. If it reduces both noise and bias, it functions as a genuine amplifier of underlying judgment quality. The evidence so far suggests it does both, in domain-specific ratios that depend on the task and the interaction pattern.
Origin
The book was conceived by Kahneman in the 2010s as a response to what he increasingly recognized as a blind spot in the original heuristics-and-biases framework: the program had focused on systematic bias and underplayed random noise. Kahneman's collaborations with Olivier Sibony (a consultant and management scholar) and Cass Sunstein (a legal scholar) brought domain expertise in organizational judgment and legal decision-making.
Publication by Little, Brown Spark in May 2021 attracted substantial attention, though the reception was more mixed than Thinking, Fast and Slow's. Critics argued that the noise-bias distinction was less sharp than the book suggested, and that noise reduction through algorithms carries risks the book underemphasizes.
Key Ideas
Noise as distinct error category. Random variability in judgment is a separate source of error from systematic bias, and requires different remediation strategies.
Algorithm consistency. Simple algorithms applied consistently outperform expert judgment across domains, largely by eliminating noise rather than by superior reasoning.
Occasion noise. The same judge reaches different conclusions at different times due to irrelevant factors (mood, weather, time since lunch) — and professional judgment exhibits substantial occasion noise.
The compression trade-off. Noise reduction shrinks the distribution of outcomes, reducing both catastrophic failures and exceptional successes — a trade-off that is beneficial in some domains and harmful in others.
Noise in the AI discourse itself. Different observers reach different conclusions from identical AI demonstrations — a fact the book's framework helps distinguish from genuine evidence-based disagreement.
Debates & Critiques
Whether noise is truly separable from bias, and whether the book's remediation proposals (decision hygiene, mediating assessments protocol) are adequate to the challenge, remains contested. Some critics argue that the book underestimates the value of judgmental variability as a source of adaptation and creativity. Others argue that the book overestimates the feasibility of algorithm substitution in domains where context sensitivity matters.
Appears in the Orange Pill Cycle
Further reading
- Kahneman, Daniel, Olivier Sibony, and Cass Sunstein, Noise: A Flaw in Human Judgment (Little, Brown Spark, 2021)
- Meehl, Paul E., Clinical versus Statistical Prediction (University of Minnesota Press, 1954)
- Dawes, Robyn, 'The Robust Beauty of Improper Linear Models in Decision Making' (American Psychologist, 1979)
- Kahneman, Daniel, Thinking, Fast and Slow (Farrar, Straus and Giroux, 2011)