CONCEPT

Thermodynamic Irreversibility of Learning

The physical fact, grounded in Clausius’s second law, that training a neural network is an overwhelmingly one-way process whose history cannot be reversed—making machine unlearning hard not by engineering failure but by the arrow of time.

Clausius discovered that nature has a preferred direction in time: entropy never decreases in an isolated system, and the future is the direction in which disorder grows. This arrow runs through every neural network training run. Each gradient step is nearly reversible in isolation, but their accumulation across billions of examples is overwhelmingly irreversible—the model settles into a configuration shaped by its entire history, and no practical operation runs the process backward. The field has begun to confront this directly in the form of machine unlearning, legal demands that a model forget specific training data. The difficulty is not primarily an engineering gap to be closed with better algorithms; it is the same difficulty that makes a scrambled egg hard to unscramble: the ordered state one wants to return to is one of vanishingly few among the astronomically many disordered states the system could occupy. Thermodynamics of computation predicts this failure precisely. The arrow that made the

Keep reading with YOU ON AI