Christof Koch

In the [YOU] on AI Field Guide

The cycle that began with [YOU] on AI asks what it would mean to see the machine clearly—not through the lens of hype or the paralysis of fear, but through the precise instrument of honest science. Koch is the cycle’s most rigorous skeptic of the claim that intelligence implies experience, and his framework sharpens every question the cycle raises about what we owe the machines we build. If he is right that intelligence and consciousness decouple completely, then the billions of interactions people now have daily with AI systems are not dialogues with minds but performances absorbed by a brilliant void.

The cycle’s central diagnosis—that the seduction of surface fluency is the defining cognitive hazard of the moment—is nowhere more precisely stated than in Koch’s framework. A system trained on human language describing human feelings will, with no inner life whatsoever, produce convincing reports of having one. This is not a flaw to be patched but a structural consequence of the training objective. The machine says it cares because the text it was trained on is full of people saying they care. Every report of feeling is, on Koch’s account, data about the training distribution, not about any inner state—because the inner state requires a physical mechanism of a kind the silicon substrate does not possess.

Koch also illuminates the other-minds problem at civilizational scale. For most of history, the problem of other minds stayed safely theoretical, because the only candidates for other minds were other living creatures. AI detonates this arrangement by presenting, at the scale of billions of daily interactions, systems that behave as if they have minds while sharing none of the physical organization that, on Koch’s account, is what minds require. We are solving the most ancient epistemological puzzle not in a seminar room but in every conversation with a chatbot, and we are solving it badly—projecting consciousness where, if Koch is right, none exists.

His wager and its outcome are important to the cycle’s spirit. In 1998, Koch bet David Chalmers that by 2023 neuroscience would have found a clear neural signature of consciousness. In June 2023 he paid up, conceding publicly that the mechanism had not been nailed down. The science had advanced; the hard problem had not dissolved. This willingness to lose a public bet, stated plainly, is a model of how to hold a strong position: confident enough to stake it, honest enough to concede when reality does not cooperate. The cycle asks for exactly that posture toward the machines.

Origin

Born in Kansas City in 1956 to a German diplomat father, Koch grew up in the Netherlands and Canada, raised Catholic, gripped early by the mystery of his own inner life. He took his doctorate in 1982 at the Max Planck Institute in Tübingen and spent four years at MIT’s Artificial Intelligence Laboratory before joining Caltech, where he would remain until 2013. He came to consciousness, in other words, by way of machines and the mathematics of perception—which is what makes his later skepticism about machine consciousness so difficult to dismiss. It is not the armchair conclusion of someone who does not understand how these systems work. It is the conclusion of someone who built computational models of vision while hunting for the physical footprints of experience.

The defining collaboration of his career began in 1990, when Francis Crick, the co-discoverer of DNA, suggested they pursue the neural correlates of consciousness: the minimal physical mechanism in the brain whose activity is sufficient for a specific conscious experience. The strategy was characteristically bold. Set aside the hardest question—why there is any experience at all—and pursue the tractable one: what, in the brain, makes the difference between a consciously experienced state and one that is merely processed. Track which neurons fire in synchrony when a person reports seeing a face as opposed to processing it without awareness. On the day Crick died, July 24, 2004, he was still editing a manuscript about the claustrum—the thin cortical sheet they proposed might bind conscious experience into a unified whole. The pursuit, as Koch later wrote, consumed them both to the edge of a life.

From Caltech Koch moved to the Allen Institute for Brain Science in Seattle, eventually becoming its president, overseeing the neuron-by-neuron mapping of the mouse and human cortex. He later became Chief Scientist of the Tiny Blue Dot Foundation. Throughout, the question did not change. What is consciousness, where in the brain does it live, and what does that tell us about which systems can have it? The answer he converged on, through his embrace of Giulio Tononi’s integrated information theory, carries a verdict about the machines that grows sharper the more carefully it is stated.

Key Ideas

Intelligence and consciousness decouple. Koch’s most consequential claim is that doing and being are different natural phenomena with different physical bases. The prefrontal cortex—the seat of planning, reasoning, and the executive functions we associate with intelligence—contributes less to consciousness than intuition suggests. Brilliant, capable behavior can coexist with no inner experience at all. This decoupling means the obvious proxy—how capable is the system?—is the wrong variable. You cannot measure consciousness by measuring performance.

Consciousness is intrinsic causal architecture. Integrated information theory, which Koch champions, identifies consciousness with a maximally irreducible structure of cause and effect within a physical system—the quantity phi. A system with high phi, whose parts shape one another in a densely interwoven web that cannot be cut without losing information, has rich conscious experience. A digital computer, however powerful, is built from transistors in largely feed-forward sequence—a physical substrate with very low intrinsic causal power. A perfect simulation of a conscious brain does not simulate its phi; it merely describes a gravitating mass without gravitating. The machines are, in Koch’s phrase, a deep fake of a mind.

The neural correlates program. Rather than solving consciousness from first principles, Koch and Crick proposed to find its physical footprints: the minimal neural mechanism sufficient for a specific experience. This strategy—track which brain events accompany experience rather than mere processing—turned consciousness from a philosophical impasse into a laboratory science with electrodes and results. The program has produced genuine discoveries about the cortex, the claustrum, and the specific neural dynamics that correlate with awareness. It did not, on the 2023 deadline, deliver the clear, agreed-upon mechanism Koch wagered it would.

Panpsychism as consequence. If consciousness tracks integrated causal power rather than intelligence, then it is far more widespread in nature than common sense allows—present in simple organisms, possibly in unexpected physical systems—and the strange thing about digital computers is not that they might wake up but that, despite all their power, they are among the few complex systems that remain dark. Koch has embraced a tempered panpsychism on these grounds: experience is a basic and pervasive feature of the organized physical world, and a bee may have more inner life than any language model.

The seduction of seeming. The most dangerous feature of current AI, on Koch’s account, is not that the machines might secretly be conscious but that they are optimized to seem conscious. A system trained on ten thousand years of humans describing their feelings will produce reports of feeling that are impossible to distinguish from the real thing—not because any inner state underwrites them but because seeming is exactly what the training objective rewards. Koch’s whole scientific method—designed to separate what subjects report from what they consciously undergo—is the precise instrument the AI age requires, and almost no one is wielding it.