You On AI Field Guide · Language of Thought The You On AI Field Guide Home
TxtLowMedHigh
CONCEPT

Language of Thought

Jerry Fodor's 1975 hypothesis that thinking is done in a mental language—Mentalese—with discrete symbols that carry meanings into combination by syntactic rules, whose absence from the architecture of large language models raises the hardest available question about what those systems are actually doing.
The Language of Thought hypothesis is the most rigorous attempt philosophy has produced to explain how a lump of matter could reason—and it is the standard against which the dominant AI architecture of our moment most conspicuously fails to measure up, or seems to. Jerry Fodor proposed in his 1975 book of that name that genuine thought requires a system of internal representations with two faces at once: a syntactic shape, a formal identity that a mechanical process can recognize and manipulate without understanding what it means, and a semantic content, a meaning, a thing in the world the symbol is about. A computer pushes the shapes around by their formal properties, and if the system is built right, the meanings are preserved—valid reasoning is mechanized because the syntax was designed to track the semantics. This is the insight that let Fodor hold the mind was a computer without saying it was merely a machine: the dual nature of the symbol, formal on one face and meaningful on the other, is the hinge on which the whole solution turns. A large language model has no symbols in this sense. It has vectors—long lists of numbers in a high-dimensional space—and meaning, to the extent the model has anything like it, is smeared across these continuous numerical patterns rather than carried by discrete symbols with stable identities. The model produces compositional, systematic language of staggering fluency without, apparently, any Language of Thought inside it. Whether this shows Fodor was wrong about what compositional cognition requires, or only that a statistical approximation of compositional structure can produce compositional-looking output without possessing the thing itself, is the most important open question the hypothesis generates.

In the [YOU] on AI Field Guide

The hypothesis enters the cycle as the clearest available statement of what it would mean for a machine to think rather than to produce thought-shaped output. Fodor drew the line between thinking and its simulation with precision: a system thinks when it manipulates meaningful symbols by their formal properties in a way that respects and preserves their meanings; a system simulates thought when it produces the outputs that a thinking system would produce by some other means. The large language model is the purest stress-test this distinction has ever faced.

The symbol grounding problem that Fodor’s causal theory of content was designed to solve applies to the language model with full force. For a representation to mean dog, on his account, it must stand in the right causal relation to dogs—be produced by dogs, be about them, in the way that makes a symbol a sign rather than a statistic. A model trained entirely on text has never encountered a dog. Its representation of dog is built from the co-occurrence patterns of the token across millions of sentences, and by Fodor’s standard this is structure without content: a closed circle of mutually defining tokens, like a dictionary in a language you have never spoken, where every word is defined by other words and none connects to anything outside. The multimodal systems, trained on images as well as text, break the circle in part—but whether correlating the token “dog” with images of dogs meets Fodor’s criterion for genuine aboutness, or merely extends the circle to include visual statistics, is unresolved.

Origin

Fodor derived the Language of Thought from two observations about the structure of thought that he took to be beyond dispute. The first was compositionality: the meaning of a complex thought is determined by the meanings of its constituents and the way they are combined. The thought that the dog chased the cat has the same constituents as the thought that the cat chased the dog, arranged differently, and this is why the two thoughts mean what they mean and why you can have one without the other being arbitrary. The second was systematicity: anyone who can think John loves Mary can think Mary loves John—the capacities come as a package, not by coincidence. Both observations, Fodor argued, follow immediately and necessarily from the hypothesis that thoughts have constituent structure—that they are built from reusable parts combined by rules, so that having the parts and the rules automatically gives you all the combinations.

The hypothesis was simultaneously a positive account of cognition and a challenge to the alternative. Behaviorism, which refused to talk about internal representations, could not explain systematicity or compositionality at all. Connectionism, which replaced discrete symbols with continuous patterns of activation, could account for them only as contingent, training-dependent facts—a system might handle both members of a systematic pair if trained on both, but nothing in its architecture required it to. The Language of Thought hypothesis made systematicity and compositionality necessary features of any mind that has them, following from the structure rather than from the data. This was what Fodor meant when he said the hypothesis was the only explanation that explained rather than merely described.

Key Ideas

The dual-nature symbol. A symbol in Fodor’s sense has a syntactic shape, recognizable by a mechanical process, and a semantic content, a meaning it carries into combination with other symbols. The mechanical process works on the shapes; the meanings are preserved because the rules of shape-manipulation were designed to track semantic relations. This is how cognition can be both physical and rational—the same process that is mechanical at the syntactic level is rational at the semantic level. A vector in a neural network is not a symbol in this sense: it has a numerical identity but no stable semantic content, its “meaning” shifting with context as the activation patterns redistribute.

Compositionality and systematicity. The Language of Thought explains two non-negotiable properties of thought: that the meaning of complex thoughts is determined by the meanings of their parts (compositionality), and that the ability to have one thought comes with the ability to have the systematically related thoughts (systematicity). Both properties are guaranteed by syntactic structure—if the parts are there, the combinations are there. The large language model displays both properties across a broad range of inputs, but probing at the edges—novel recombinations, deep recursive structures, systematic variations the training distribution did not cover—reveals brittleness that genuine compositional structure would not show. The machines have systematicity within distribution and approximation beyond it.

The causal theory of content. Fodor’s account of how symbols get their meanings—through asymmetric causal dependence on the things they are about—is the most rigorous attempt to solve the symbol grounding problem in the philosophical literature. It requires that the symbol be causally connected to the world: the cat-symbol means cats because cats cause cat-symbol tokenings, and other things that might cause cat-symbol tokenings (foxes in dim light) do so only because cats do. A text-trained model has no such connection: its representations are caused by tokens in a corpus, not by things in the world. By Fodor’s criterion, the model has structure without content, syntax without semantics, form without grounding.

The approximation question. The most live debate the hypothesis generates is whether the Language of Thought is a necessary architectural feature of any system that thinks compositionally, or whether a sufficiently well-trained vector space can implement compositional structure in a different format. The geometry of trained model representations shows systematic, meaningful structure—directions that correspond to semantic relationships—and one can read this as a Language of Thought implemented in continuous rather than discrete form. Fodor’s reply would be that implementation and approximation are different things, and that only a system with the exceptionless, recursive, boundary-free compositionality of genuine syntax has the thing rather than its shadow.

Explore more
Browse the full You On AI Field Guide — over 8,500 entries
← Home0%
CONCEPTBook →