CONCEPT
Kelvin’s Creed
The doctrine—first laid down in Kelvin’s 1883 address to the Institution of Civil Engineers—that knowledge unexpressed in number is meager and unsatisfactory, which became the founding philosophy of the AI benchmark culture and carries within it, inseparably, the warning of its own misuse.
When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.” William Thomson, Lord Kelvin, delivered this sentence to the Institution of Civil Engineers in 1883, and it has haunted science and engineering ever since. It is both a genuine discipline—the insistence that vague claims be made precise enough to test—and a trap that Kelvin himself fell into: he measured the Earth’s heat flow with impeccable rigor and got the age of the Earth wrong by a factor of more than a hundred, because he measured what was accessible rather than what governed the answer. In the age of artificial intelligence,
Kelvin’s creed is instantiated at planetary scale in the
benchmark culture of machine learning, where every model’s worth is established by its