Constant Conjunction and the Correlation Engine

EDO SEGAL: I want to open this round with a confession instead of a question, because the best questions I know come out of wounds. I started in Assembler. I was raised by the machine code, and for fifty years using a computer meant translating my intention into its grammar and paying a tax on every conversion. In the winter the machines crossed over, that tax went to zero — the box met me in my own messy language. And one night, late, I described to it a system I could half-see, and it didn't just complete my sentence. It told me why my approach would fail — that a change here would cause a failure there. It reasoned about a cause. Or it produced the exact shape of reasoning about a cause, and I could not tell the difference, and I still can't. Mr. Hume, you say that what I met was constant conjunction wearing the costume of insight. Take that apart for me. Slowly.

HUME: I'll take it apart gently, because the gentleness is the point. What happened to you is real, and I do not for a moment deny that the machine produced something useful and true. But consider what you actually observed. You described a structure; the machine returned a continuation in which the words this change were followed by the words that failure — because in the vast text on which it formed its habits, descriptions like yours were followed, constantly, reliably, by descriptions like that. It did precisely what I claim you do when you watch the billiard balls: it registered a conjunction and was carried, by habit, from the one to the other. The feeling of necessity — the sense that the change must produce the failure — was supplied, as it is always supplied, not by anything in the data but by the determination of a mind exposed to repetition. The only difference is that the determination this time occurred in silicon and was handed back to you, and your mind, my dear Edo, did the rest. You felt a cause being understood because you are a creature who cannot encounter a fluent account of a cause without feeling that. For a hundred thousand years, fluent talk of mechanism came only from a mind that grasped mechanism, and your interpretive faculty has no defense against the first machine that ever produced the talk without the grasping.

PEARL: May I take the other side of the wound?

EDO SEGAL: That's why you're here.

PEARL: Edo, ask the question Hume keeps stepping past. Why did it work? Not why did it sound right — why was it actually, usefully right about your system? Hume says: because the conjunction was in the text. Fine. But notice what that requires. For the machine to reliably tell you that this change causes that failure across the endless space of systems it has never seen, the text it learned from must be lawful — and it is lawful because it was written by people who had causal models, who knew that this change produces that failure, and whose knowledge left its trace in the regularities of what they wrote. So I will grant Hume the whole mechanism and still defeat his conclusion. The machine has done something genuinely impressive: it has reconstructed, from the wake, a shadow of the boat that made the wake. Hume calls that constant conjunction and stops. I call it a compression of other people's causal models — secondhand, ungrounded, and brittle exactly where the borrowed models run out. The machine did not grasp why your change fails. It inherited, statistically, the grasp of ten thousand engineers who did. And the moment your system departs from anything any of them ever wrote down, the inheritance is exhausted and the fluency continues without it. That is the tell. Push it into genuine novelty — an intervention no text described — and watch the shadow of causal reasoning detach completely from the substance, while the confidence never wavers.

HUME: But Dr. Pearl, notice the move you just made, because it is the move your entire school makes. You said the machine inherited "the grasp" of ten thousand engineers who "knew" the cause. I ask you my standing question. What did those engineers have, when they "knew" it, that was not itself constant conjunction, formed over their own careers of watching this-change-followed-by-that-failure? You speak of their causal models as if you had seen one — as if a model of mechanism were a thing you could point to in the engineer's head, distinct from the long habituation of experience. Show me the impression. When the senior engineer feels that the change will break the system, what is that feeling, examined honestly, but the same determination of the mind I described, produced by the same repetition, only housed in meat? You want the human to have a boat and the machine to have only the wake. I am telling you it is wakes all the way down, and the conviction that you possess a boat is precisely the illusion my philosophy was written to dissolve.

PEARL: No. And here is exactly where I plant my flag, because this is the crux of three centuries. The engineer has something the machine provably lacks, and it is not a feeling — it is a capacity, and it is testable. The engineer can answer a question the machine cannot: what would happen if we changed the deployment in a way no one has ever tried. He can run the world forward in his head under an intervention that exists in no text, because he is not carrying around a table of conjunctions — he is carrying a structure of mechanisms, arrows that say which thing produces which, and he can reach in and cut an arrow and recompute. You say show me the impression. I say: the impression of a causal model is the ability to answer interventional and counterfactual questions correctly about cases you have never seen. That is the operational signature, and the engineer passes it and the machine fails it, systematically, the instant you leave the distribution. The difference between us is not a feeling I cannot reduce. It is a competence I can measure.

HUME: Then we have found the seam, and I want to mark it carefully, because it is the seam of the whole evening. You define the possession of a model as the ability to answer questions about cases never seen. But the human answers those questions also by induction — by analogy from cases he has seen, carried across by habit to the case he has not. When your engineer imagines the untried intervention, he is not consulting an oracle of mechanism; he is running a simulation built entirely from past conjunctions, and projecting. It is a richer projection than the machine's, I grant you, longer-trained and better-calibrated. But it is projection, and it is fallible in exactly the way mine is — it fails when the world stops resembling the cases that built it. You have not described a different rung. You have described a taller pile of the same rung.

EDO SEGAL: Let me hold you both there, because something just happened that I need to name for the reader, who can't see your faces. That was the first exchange tonight where neither of you was being gracious. Mr. Hume says the model is an illusion — a taller pile of conjunctions. Dr. Pearl says the model is a measurable competence — answer the intervention question or you don't have one. And here is what I notice: you have not actually disagreed about the machine. You both agree it cannot answer the untried-intervention question. You disagree about whether the human really can, or only flatters himself that he does. The fight is not about silicon. It's about the meat. Which means the next round has to be about the child pushing the cup off the high chair — because Dr. Pearl, that child is your whole case that the meat does something the machine doesn't, and Mr. Hume, I want to watch you take the cup away from her.