CONCEPT
Overconfidence and the Calibration Problem
Tversky and Kahneman's finding that people assign probabilities to their judgments that systematically exceed actual accuracy — a calibration failure that AI's smooth output makes worse by decoupling surface cues from underlying accuracy.
Overconfidence is the systematic miscalibration of judgment in which the probabilities people assign to the correctness of their answers exceed the frequency at which those answers actually prove correct. Events judged 90% certain occur approximately 75% of the time; events judged certain sometimes fail to occur. The bias is robust across populations, domains, and expertise levels. In the AI context, overconfidence produces a specific calibration problem: the normal cues that calibrate confidence — effort expended, difficulty encountered, frequency of errors — are decoupled from accuracy when the source is an LLM. A hallucination arrives with the same fluency as an accurate statement. The surface cue of effortless polish no longer tracks the underlying quality, and the calibration system has no basis for distinguishing them.
In The You On AI Field Guide
Tversky's work on overconfidence, conducted with Kahneman and Baruch Fischhoff through the 1970s and 1980s, established that calibration errors are not random