Demis Hassabis vs Hubert Dreyfus on AI · Ch8. The Background a Machine Cannot Inhabit ← Ch7 Ch9 →
Txt Low Med High
HOUR TWO — THE TOOL AND THE BACKGROUND
Chapter 8

The Background a Machine Cannot Inhabit

Page 1 · The Background a Machine

**EDO SEGAL:** Hubert, define the background, and then tell me the thing you've had to concede about it — because you're the rare critic who's actually updated, and I want the reader to see a philosopher change his mind in real time.

**DREYFUS:** The background is the vast, tacit, culturally constituted fabric of shared practice against which every explicit thought becomes possible. You know a restaurant is not a place to lie down on the floor. You know "Can you pass the salt?" is not a question about your arm's range of motion. You know a colleague who says "Fine." in a certain tone is not fine. None of this is stored as a rule you retrieve. It's a familiarity, an ability to cope, absorbed by living a human life among other humans — a know-how, not a stock of know-that. And I argued for fifty years that it cannot be formalized, and the argument was a regress: any attempt to write down the rules of common sense must use terms whose meaning depends on further background, which must then be made explicit, and so on without end. The background is the condition of meaning, so it can't be assembled out of meaningful pieces without presupposing itself. That's why Lenat's Cyc project, which tried to hand-code common sense as millions of explicit assertions, consumed decades and a fortune and did not produce common sense. The wall was exactly where I said it would be.

· · ·
Page 2 · The Background a Machine

Now the concession, and I'll make it cleanly because intellectual honesty demands it. The new systems do not try to formalize the background. They never write down a single rule of common sense. Instead they absorb the background's *textual residue* — when millions of people write about restaurants, their writing presupposes that one doesn't lie on the floor, and a model trained on that writing learns the statistical shape of the presupposition. So Cyc failed because it tried to formalize the background; the large language models succeed, to a remarkable degree, because they don't — they approximate it from the data the background has shaped. I said the background could not be made explicit. I was right. I did not foresee that a machine might capture much of it *without ever making it explicit*, by learning from the traces of beings who possessed it. On that specific point, the empirical edge of my prediction was beaten, and any honest man says so plainly.

**EDO SEGAL:** Demis, that's a real concession. Does it give you the whole thing?

· · ·
Page 3 · The Background a Machine

**HASSABIS:** It's a generous concession and I want to honor it by not overclaiming. No — it doesn't give me the whole thing, and I'd be doing exactly what the professor accuses my field of if I said it did. Here's what I think it gives me, precisely. It gives me that the background, or enough of it to be enormously useful, is *recoverable from its traces* — which is a deep fact about the world, that embodied practice leaves a learnable shadow dense enough to reconstruct most of the practice. That's not nothing; it's most of why these systems work at all. What it doesn't give me, by the professor's own argument and I think he's right, is the guarantee that the reconstruction holds at the edge — in the genuinely novel situation, far from the traces, where only something that actually inhabits the practice can improvise. So my honest position is: the machine has the background's shadow, the shadow is shockingly good, and the shadow thins exactly where the professor says it does. Where we still disagree is how much of human life happens in the thick middle, where the shadow is plenty, versus the thin edge, where it fails. I think the middle is most of life. He thinks the edge is where everything that matters happens.

· · ·
Page 4 · The Background a Machine

**DREYFUS:** That's the fairest statement of the disagreement anyone has made, and I'll sharpen it rather than soften it. The reason the edge matters disproportionately is that the edge is where *understanding earns its keep*. In the thick middle, you don't need understanding — pattern is enough, and the machine has the pattern, and I concede the machine should do the middle and probably do it better than tired humans. But we don't keep experts around for the middle. We keep them for the edge: the novel disease, the unprecedented case, the situation no protocol anticipated, where the whole depth of an embodied background gets improvised into a response no rule dictated. And the machine meets the edge by extrapolating from regularities, and when the situation lies far enough from them, the extrapolation fails — not randomly, but *plausibly*. The failures look right. They read well. They'd pass a glance. They are wrong in ways only a person with real background can detect. And that — the plausible failure at the edge — is not a bug to be patched out. It is the structural signature of a system that processes the pattern without inhabiting the involvement the pattern presupposes. The hallucination is the background problem made visible for one second before the fluency papers over it again.

**HASSABIS:** But "the edge is where everything matters" is a claim I can pressure with the protein database. AlphaFold's value is overwhelmingly in the thick middle — two hundred million structures, the vast routine of biology, that researchers now get for free instead of grinding out by hand. The edge cases where it's unreliable are real and known and flagged, and the working biologist treats it exactly as the professor wants, as a powerful instrument she checks at the boundary. The middle it automates is not trivial. It's most of the actual labor of a science, freed up so the humans can spend themselves on the edge. That's not the machine pretending to inhabit the practice. That's the machine doing the part of the practice that didn't need inhabiting, so the inhabited part gets more human attention, not less.

· · ·
Page 5 · The Background a Machine

**DREYFUS:** And if that's how it's used — instrument in the middle, human at the edge, the boundary respected — then we have far less quarrel than the framing suggests, and I'll say so. My fear was never the tool used that way. My fear is the *drift*: that a tool this good in the middle trains us to trust it at the edge too, that the boundary erodes one unexamined acceptance at a time, and that the human who was supposed to hold the edge has, through years of leaning on the middle, lost the background that let her hold it. The danger isn't the machine overstepping. It's us under-standing, in the literal sense — ceasing to stand under the work, to bear it up with our own understanding, because the tool stood under it for us until we forgot how. That's the [drift toward the frictionless](https://www.youonai.ai/fieldguide/med/aesthetics_of_the_smooth), and it's not the machine's doing. It's ours.

**EDO SEGAL:** Hold there, because you've just handed me the bridge to the next round and you did it from opposite sides. Demis says: used rightly, the tool frees the human for the edge. Hubert says: the danger is that the rightness erodes, and we drift. The whole argument about timelines — about how fast this is coming and whether it ever reaches the general thing — turns on which of those trajectories is real. And there's a fallacy Hubert named decades ago that the entire field keeps committing. After the break: the tree, the moon, and the first step.

· · ·
Continue · Chapter 9
The Tree, the Moon, and the First Step
← Prev 0%
Ch8 Next →