CONCEPT
Conservation of Information
The Lavoisierian principle applied to computation: a model cannot output information it did not, in some form, take in, and every apparent creation is a transformation of what was already present—which is simultaneously a deflation of AI mystique and a precise description of where AI’s genuine value actually lives.
Conservation of information is the computational echo of
Lavoisier’s axiom that nothing is created and nothing destroyed—that every transformation is a rearrangement of conserved substance, not an act of creation from nothing. Applied to machine learning, the principle holds that a model cannot generate genuine novelty that was not, in some form, present in its training data: the brilliant essay it produces is a transformation of the corpus that trained it, the pattern it “discovers” was latent in the data, and the answer it cannot give to a question about events after its training cutoff is simply absent from the system because the information was never put in. The no-free-lunch theorems in learning theory give this intuition its mathematical form: no algorithm outperforms all others across all possible problem distributions; performance is redistributed by the assumptions the algorithm embeds, never manufactured from nothing.
Pearl’s critique of curve-fitting