Measurement Validity (AI)

Measurement validity is the gap between the number and what the number is a number of—the question that distinguishes a science from a more precise form of confusion. Lavoisier succeeded where the phlogiston chemists failed not because his instruments were more sensitive but because he measured the right quantity: he weighed the gases others had let escape as if they were nothing, and by doing so showed that burned metals gain mass rather than lose it, which the reigning theory could not survive. In AI evaluation, the equivalent failure is endemic: a system that scores ninety percent on a reasoning benchmark may or may not reason in any meaningful sense, because the benchmark measures performance on a fixed distribution of questions and the inference of general reasoning capability is an additional claim the test was never designed to certify. Goodhart’s Law names the specific pathology when models are trained, directly or indirectly, on the distribution a benchmark tests: the score detaches from the