CONCEPT

The Space of Possible Texts

The totality of all character-string permutations that can be formed from a given alphabet—Borges’s Library of Babel reformulated as a mathematical object, which turns out to be precisely the space that a large language model learns to navigate by imposing a probability landscape that distinguishes the plausible from the gibberish.

The Library of Babel (1941) describes a universe that contains every possible permutation of characters drawn from an alphabet of twenty-five symbols, organized into books of fixed length. The library therefore contains every possible text: every truth, every lie, every masterpiece, every string of gibberish, all indistinguishably shelved together. Borges understood the combinatorial structure with mathematical precision, and his story is among other things a meditation on the difference between having access to all possible texts and being able to find, use, or trust any of them. A large language model navigates a close relative of this space. The model does not store the library; nothing could. What it learns is a probability distribution over the space—a landscape of plausibility imposed on the flatness of all possible sequences, assigning high probability to sequences that appear in human writing and low probability to the

Keep reading with YOU ON AI