CONCEPT
The Calibration Problem (Daston)
The structural difficulty of developing accurate user models of a knowledge technology's reliability — and the specific way AI impairs both conditions (error detection and error identification) on which calibration historically depends.
Calibration is the process by which users of a knowledge-producing technology learn to assess its outputs with accuracy proportionate to the outputs' actual reliability — extending trust where warranted and withholding it where not. It develops through a specific mechanism: the user produces or receives an output, compares it against some independent source of information, discovers a discrepancy, and updates her model of the technology's reliability. Over time, through repeated encounters with discrepancies of different types, the user develops a calibrated intuition — a sense, not always fully articulable but operationally reliable, of when the technology can be trusted and when it cannot. The mechanism depends on two conditions: errors must be detectable (independent information exists against which outputs can be compared) and identifiable (errors can be distinguished from accurate outputs, carrying markers that signal unreliability).
In The You On AI Field Guide
The calibration mechanism has been studied in detail across the history of scientific instrumentation. The microscopist