The Principle of Least Effort

Zipf's principle of least effort is the claim that human beings, in solving their immediate problems, view them against the background of their probable future problems and strive to minimize total effort across the whole anticipated horizon. Applied to language, it generates a precise and elegant account of Zipf's law: speech is the site of a permanent tug-of-war between the speaker's economy (minimize vocabulary size, use common words for everything) and the listener's economy (minimize ambiguity, demand a unique word for every distinct meaning). The Zipfian distribution of word frequencies is the equilibrium of these two opposing forces. The principle resonates with the AI age in ways Zipf could not have anticipated: modern tokenization algorithms are the principle of least effort rendered as code, assigning cheap short symbols to common patterns and expensive decompositions to rare ones. The training objective of a large language model—predict the next token as accurately as possible—pressures the system to allocate