Understanding Is Compression

The single most useful idea Gregory Chaitin contributed to the study of minds and machines is also the simplest to state: understanding is compression. To comprehend a body of data is to find a theory shorter than the data that generates it—a law, a model, a program—from which the data can be regenerated. Newton did not memorize the positions of the planets; he found a handful of equations from which all those positions follow. The equations are vastly shorter than the data they predict, and the compression is the content of the understanding. Chaitin made this precise in the framework he founded simultaneously with Andrei Kolmogorov and Ray Solomonoff: the complexity of a thing is the length of its shortest program, and to understand the thing is to possess that program. A phenomenon that cannot be compressed at all—whose shortest description is the phenomenon itself—is incompressible and therefore, in principle, beyond understanding, because there is