What Cannot Be Computed

In the AI Story

The AI discourse has systematically confused cleverness with intelligence, and the confusion serves specific interests. If intelligence is cleverness, AI is intelligent, because AI is clever. If AI is intelligent, it can replace human intelligence, because it is the same kind of thing, only faster and cheaper. The chain of reasoning from 'AI is clever' to 'replace the human' passes through the definition of intelligence as cleverness, and the definition is the link Midgley spent decades trying to break.

Breaking it requires insisting on the distinction between the computational and the experiential — between what can be computed and what cannot. Computation can process the rules of chess. It cannot experience the thrill of a brilliant sacrifice. Computation can generate a poem that follows the formal properties of a sonnet. It cannot experience the struggle of finding the word that means what you feel but have never articulated. Computation can produce a medical diagnosis that is statistically optimal given available data. It cannot experience the compassion a good doctor brings to the delivery of bad news — the awareness that the patient is a person, not a data point.

These experiential qualities are not decorations applied to computational outputs. They are constitutive of what makes the outputs meaningful. A chess game played by two conscious beings who experience tension, risk, and satisfaction is a different kind of event from a chess game played by two algorithms. The moves may be identical. The event is not. The difference is the presence or absence of experience, and the presence of experience is what transforms a sequence of optimised moves into a game — something that matters to the beings who play it.

The practical implication converges on a single point. The aspects of human intelligence that cannot be computed are central, not marginal. They are constitutive of what makes intelligence valuable — not instrumentally valuable (useful for solving problems) but intrinsically valuable (constitutive of a life worth living). The child who asks 'What am I for?' is exercising precisely this non-computational capacity. Her question cannot be computed — not because computation is not powerful enough, but because computation is the wrong kind of process. Computing and caring are different operations.

Origin

The framework is developed across Midgley's philosophy of mind writing, particularly in Beast and Man (1978), Wisdom, Information and Wonder (1989), and Utopias, Dolphins and Computers (1996). It draws on the Aristotelian tradition's distinction between kinds of activity (phronesis, techne, theoria) and on phenomenological work on the irreducibility of experience.

Key Ideas

Computation vs. experience is categorical. The difference is not degree but kind — no amount of computational sophistication will produce experience.

Caring cannot be computed. Caring requires being a creature with stakes; computation has no stakes and cannot acquire them by scaling.

The central aspects of intelligence are non-computational. Significance, meaning, moral agency — the aspects that make intelligence valuable — are exactly the aspects computation cannot reach.

Function is not life. A process that produces outputs without experiencing them is a function; life is what experiences its own activity.