Experimental Epistemology

In the [YOU] on AI Field Guide

The cycle that began with [YOU] on AI returns to experimental epistemology as the discipline most urgently needed by the field that cannot read its own systems. We have built, in the large neural network, a knowing thing whose knowledge is locked inside billions of numerical weights, none of which announces its meaning. The network classifies, predicts, reasons—and when we ask how, we have, for most of the field's history, had no answer beyond "the training produced these numbers and the numbers work." This is an intolerable situation for anyone who shares McCulloch's conviction that knowing is a physical process open to investigation. If the knowledge is in the structure, then the structure can be probed. The discipline that has grown up to do the probing is mechanistic interpretability, and it is experimental epistemology under a new flag.

The parallel is not loose. When McCulloch and Lettvin pushed an electrode into the frog's optic nerve, they were asking a single, sharp question: what does this cell respond to? What feature of the world makes it fire? Interpretability researchers ask the identical question of artificial neurons. They find the inputs that maximally activate a given unit in a trained network, and from the pattern they infer what the unit has learned to detect. A unit that fires for images of wheels; a unit that responds to the syntactic structure of a question; a unit that, across many contexts, tracks whether a statement is true or false. These are the bug detectors of the machine, mapped by the same logic McCulloch used on the frog: behavior recorded, stimulus varied, function inferred.

The concept connects directly to McCulloch's broader program: his conviction that there is no magic in knowing, only structure we have not yet understood. The dream of interpretability is to make the machine's knowing legible enough to trust or to correct—to find where in the structure the knowing lives, to read what the machine's eye tells the machine's brain. Whether the dream is achievable at the scale of trillion-parameter systems is the open question. That it is the right dream, McCulloch settled in 1959.

Origin

McCulloch arrived at experimental epistemology through the same question that organized his entire career: how does a physical object—a brain, three pounds of electrified tissue—come to contain knowledge of an abstraction like number? The question demanded a physical answer, which meant a laboratory answer. He trained in philosophy at Yale and medicine at Columbia precisely because neither alone was sufficient: philosophy could pose the question, medicine could open the system, and the combination might produce an answer.

The 1959 frog study was the method's full realization. The team placed electrodes on the frog's optic nerve and presented various stimuli, discovering that different fiber types responded to categorically different features. The most celebrated finding was the bug detector: a fiber type responsive not to light or general shapes but to small, dark, convex objects moving across the visual field. The epistemological implication was seismic: the frog's nervous system does not record reality; it imposes its own categories upon reality, categories shaped by the frog's evolutionary needs and built into its retinal anatomy. The categories of perception are physical structures. What the frog can know is determined by what its eye is wired to extract. Epistemology, read McCulloch, had been found in tissue.

Key Ideas

The knower shapes the known. The central finding of the frog study is that perception is not passive registration but active selection. The frog's retina does not transmit the world; it filters, selects, and pre-digests the world according to what the frog needs. The categories by which the frog carves up its visual field are built into its anatomy by evolution. Applied to artificial systems: a trained neural network is precisely a system whose way of carving up the world is built into its structure—into the patterns its layers have learned to detect. The early layers of a vision network learn edges and gradients; later layers learn textures, then parts, then objects. These are the bug detectors of the machine, learned rather than evolved, but identical in principle: feature detectors tuned to extract specific regularities, stacked into a hierarchy that transforms raw input into something the system can act on.

The risk of projection. McCulloch's method carries its own danger, which he embodied precisely: when his team called a fiber a "bug detector," they were imposing a human category—bug—onto a frog's nervous system that knows nothing of bugs. The cell detects a pattern of moving contrast; bug is the experimenters' interpretation of why that pattern matters. The same risk haunts mechanistic interpretability, far more acutely. When a researcher declares that a particular artificial neuron "represents truth" or "encodes the concept of a dog," they may be projecting a human concept onto a pattern of activations that does not honor the boundaries of that concept at all. The network's actual categories may cut the world along seams no human word captures. Rigor, in experimental epistemology, is the relentless testing of one's interpretation against the system's behavior—varying inputs, checking whether the proposed meaning predicts the unit's firing across new cases, discarding the story when the system refuses to confirm it.

From frog to GPT. The logic of experimental epistemology scales, in principle, from the frog's retina to the transformer's internal representations, but the scaling introduces qualitative difficulties. The frog's detector was a single cell type in a circuit whose function was clear from evolutionary first principles: the frog must eat bugs to live. An artificial neuron's "detector" was learned by gradient descent on a distribution of text or images, with no evolutionary context to constrain interpretation. The features may be polysemantic—a single unit responding to multiple unrelated concepts that merely happen to co-occur in the training data. The circuits are more numerous, more deeply stacked, and more interdependent. McCulloch's method is necessary; it is also more ambiguous in its application, and humility about that ambiguity is part of the legacy.