CONCEPT
Fit vs. Truth
Kalman’s hardest insistence: a model that matches observed data has demonstrated fit, not truth—and the gap between them is not a technical shortcoming to be closed by more data but a structural feature of learning that opens precisely at the situations no dataset covered.
The distinction Rudolf Kalman pressed throughout his career, and most urgently in his final decades, is deceptively simple and structurally demanding: a model can fit every data point it was trained and tested on, generalize beautifully within its distribution, and still be the wrong model—one that misrepresents the mechanism that generated the data rather than capturing it. His
Kalman filter is optimal with respect to a model of the system, and if the model is wrong, the filter optimally tracks a fiction while reporting confident estimates it has not earned. The same structure governs every machine learning system: low cross-entropy on held-out data demonstrates that the system fits the sampling distribution; it does not demonstrate that its internal representation corresponds to the true structure of the world. The gap opens at the long tail—the rare, the novel, the out-of-distribution—where fit and truth diverge and no held-out set provides the test.
Judea