David Hume vs Judea Pearl on AI · Ch10. The Death Cross and the Apprentice ← Ch9 Ch11 →
Txt Low Med High
HOUR THREE — THE LEDGER AND THE COUNTERFACTUAL
Chapter 10

The Death Cross and the Apprentice

Page 1 · The Death Cross and
Software Death Cross
Software Death Cross

EDO SEGAL: Let me set this round with numbers, because it's about what the numbers mean. By early 2026 a trillion dollars of market value had left the public software industry — the software death cross, the moment machine capability visibly crossed the cost of the humans who used to supply it. Entry-level programming jobs down sixteen percent. And I sat in board meetings, quarter after quarter, where the arithmetic was simple: if five amplified people do the work of a hundred, why pay for a hundred? I want to put my own confession on the table first — I kept the hundred, and bet that a hundred amplified people building more ambitious things beats five building the old things cheaper, and I know the structure punishes that choice. So here is the round. Mr. Hume, Dr. Pearl — the machine is replacing the apprentice. The junior who used to write the broken code and learn why it broke. What is being lost, and does either of your philosophies tell us whether it survives?

· · ·
Page 2 · The Death Cross and

PEARL: Mine tells us exactly what is lost, and it is the thing I have spent my life on, so forgive the intensity. The apprentice is not a cheaper unit of production. The apprentice is a creature climbing the ladder by doing — writing the code that fails, and in the failing, intervening on the system and watching the consequence, and from a thousand such interventions building a causal model of why software breaks that no amount of reading finished code could supply. The junior is the child with the cup, and the codebase is the high chair. Now watch what the machine does. It hands the junior finished code — correct, fluent, the wake of ten thousand seniors' models — and the junior consumes the output of causal understanding without ever performing the intervention that builds it. We are removing the cup from the high chair. We are producing a generation that has read every answer and pushed nothing off the table, and they will be, in the precise sense I have defended all night, first-rung practitioners: fluent, fast, and unable to reason about the novel break, because they never built the model, because the model is built only by doing and we automated the doing. And here is the tragedy of time horizons: the forge works on a decade's delay. Remove the friction now, and you discover the missing seniors in ten years, long after the executive who booked the savings was promoted. Every firm behaves rationally and the profession is strip-mined of its future. This is not an economic prediction. It is the ladder, applied to a career.

· · ·
Page 3 · The Death Cross and

HUME: I agree with the structure of Dr. Pearl's worry and I want to deepen it with mine, because custom is the thing being interrupted and custom was my whole account of how a mind is made. Recall that I hold the foundation of cognitive life to be not reason but custom — the great guide — the principle by which repeated experience shapes our expectations below the level of argument. The expert does not reason her way to expertise; she is habituated into it, by the slow pressure of ten thousand cases, until her dispositions track the domain. Dr. Pearl calls this building a causal model; I call it the sculpting of custom; and for once the dispute does not matter, because we agree on what destroys it. The apprentice's expertise is custom forming under the pressure of repeated, consequential experience — and the machine removes the experience while delivering the result. The junior gets the conclusion without the habituation, and a conclusion without habituation is, in my strict sense, not knowledge in that person at all — it is borrowed testimony, held without the internal determination that experience alone installs. So I will give Dr. Pearl's tragedy its Humean edge: we are not merely failing to build the seniors' models. We are producing minds full of true beliefs they have no custom to support — fluent in conclusions they could not have reached and cannot defend, exactly as the machine is. The apprentice is becoming the machine: a bundle of ungrounded outputs. And I do not know that it survives, Edo, because the thing that produced expertise was precisely the friction we are so proud of removing.

The kid in Lagos, the kid in Trivandrum, who never had a senior to apprentice under at all — for them the alternative to the machine was not a master.

EDO SEGAL: That's the bleakest convergence yet, and I have to test it, because it sounds like two old men mourning friction. Dr. Pearl — there's a counterargument, and it's not stupid. The machine is also the most patient tutor in history. The kid in Lagos, the kid in Trivandrum, who never had a senior to apprentice under at all — for them the alternative to the machine was not a master. It was nothing. Doesn't the machine democratize the very climbing you say it's destroying?

· · ·
Page 4 · The Death Cross and

PEARL: It can, and I will not be the man who denies a child a teacher. But notice the condition, because it is the same condition as the cup. The machine helps the child climb only if it makes her do the intervention — only if it makes her push the cup, struggle with the broken code, sit in the not-knowing long enough to build the model — rather than handing her the fallen cup already on the floor. A tutor that answers before the question finishes forming does not build the apprentice; it paves the ground where the apprentice would have grown. So the technology is not destiny in either direction. Deployed to preserve the struggle, it is the greatest democratizer of the ladder ever built. Deployed to remove the struggle — which is what engagement metrics and quarterly arithmetic reward — it is the greatest destroyer. The variable is not the machine. It is whether the people deploying it optimize for the child climbing or for the child comfortable. And I have met very few ed-tech companies whose metric is "we made the learner sit longer with not-knowing."

· · ·
Page 5 · The Death Cross and

HUME: And here I must enter my one genuine disagreement with Dr. Pearl tonight, because he has been too pessimistic and my philosophy, of all things, supplies the optimism. I argued that custom, though it has no rational ground, is usually well-tuned to the world, because nature constituted us and our environment together such that our habits track reality well enough to live by. The apprentice's friction was never sacred in itself — it was merely the available channel through which custom got installed. If the machine can open a different channel — if it can engineer the repeated, consequential, interventional experience Dr. Pearl rightly demands, but faster and for more people — then it has not destroyed the forge. It has rebuilt it. I am a skeptic about reason, Dr. Pearl, not a sentimentalist about suffering. The struggle is not holy. The habituation is necessary. And whether the struggle is the only road to the habituation is an empirical question your own ladder leaves open. You said it yourself: a system that makes the child intervene builds the model. Then build that, and the apprentice survives in a form neither of us would recognize but my philosophy would entirely endorse — custom, installed by a new guide, in more minds than ever had a master.

· · ·
Page 6 · The Death Cross and

EDO SEGAL: [pause] I want to honor that, because the skeptic just out-hoped the engineer, and the reader should feel how strange that is. Dr. Pearl, the causal optimist, is the one warning the forge will die. Mr. Hume, the great doubter, is the one saying it can be rebuilt — because to him the forge was never the friction, it was the habituation, and habituation can take a new channel. [long pause] Hold both, because the worker at the kitchen table needs them both: Pearl's warning that the easy default strip-mines her child's future, and Hume's insistence that a chosen default could build it instead. Whichever is right, the decision is a decision, not a weather system. Next round, we climb to the rung Dr. Pearl says is the highest and the most human — the one the machine is furthest from, the one that begins with two words a corpse cannot say: what if. The counterfactual animal. After this.

· · ·
Continue · Chapter 11
The Counterfactual Animal
← Prev 0%
Ch10 Next →