Ensemble Knowledge

Gibbs’s most radical methodological invention was the ensemble: instead of asking what the exact state of a gas was—an impossible question given a quintillion molecules—he asked how probability was distributed across all the states the gas might occupy, and recovered the real behavior of the system as an average over that imagined cloud. He did this not because he preferred probability to certainty, but because certainty was unavailable; the ensemble was a confession of ignorance transformed into a tool. A large language model is the same kind of object: it does not store answers or retrieve facts; it learns a distribution over possible continuations of any prompt, and generates by sampling from that distribution. Its knowledge is ensemble knowledge—knowledge of the regularities in the distribution, of the shape of the cloud. What ensemble knowledge does not and cannot provide is a guarantee about any specific instance: the model