George Zipf

In 1949, a Harvard philologist published a 573-page book arguing that a single mathematical regularity governed most of human behavior. George Kingsley Zipf was wrong about the grand theory. He was right about the curve. Rank the words of any large body of text by how often they appear, and the frequency of a word falls in near-perfect inverse proportion to its rank: the most common word appears roughly twice as often as the second, three times as often as the third. The curve holds across English, Chinese, Russian, formal prose, casual speech, code. Zipf charted it by hand, with index cards, and died in 1950 before any machine could exploit what he had found. The machines came seventy years later, and the terrain they are trained on is, quite literally, his. A large language model's token distribution is Zipfian to the core: a small head of high-frequency tokens dominates the corpus, and a vast long tail of rare