Mary Shelley vs Fei Fei Li on AI · Ch8. The Death Cross ← Ch7 Ch9 →
Txt Low Med High
HOUR TWO — THE CREATURE AND THE LEDGER
Chapter 8

The Death Cross

Page 1 · The Death Cross
Software Death Cross
Software Death Cross

EDO SEGAL: Let me set the table with numbers, because this round is about what the numbers mean. By early 2026, a trillion dollars of market value had left the public software industry — the software death cross, the moment the cost of producing a capability collapses and the value of selling it collapses with it. Entry-level programming jobs down sixteen percent since 2022, the floor eroding first. And I have sat in the board meetings, quarter after quarter, where the arithmetic on the table was brutally simple: if five people can do the work of a hundred, why pay for a hundred? I kept the team. I am aware the structure punishes that choice. Fei-Fei, you have fought one word harder than any other in this whole domain — you want to replace the word replace. So you first. What is the death cross actually measuring?

Apprenticeship Problem
Apprenticeship Problem

LI: It is measuring a choice that the world is mistaking for a fact, and the mistake is the most dangerous thing in the number. Let me be precise. The death cross is real; the capability is real; the displacement is real. I will not be the person who tells the displaced worker it is a mirage. But what the market is repricing is not the machine can do the job. It is the machine can do something that resembles the job well enough that a decision-maker with a cost target will act on the resemblance. And whether that resemblance becomes augmentation or replacement is not determined by the technology. It is determined by us. The same diagnostic model makes a radiologist obsolete or makes her faster and less exhausted, depending entirely on the intention wrapped around it. So when I say replace the word replace, I am not being sentimental. I am pointing at the actual lever. The technology does not decide whether your job survives. People decide, in design and deployment and law, and they are currently deciding badly because the augmentation path is slower and the replacement path clears a fiscal year. The death cross is measuring the triumph of the quarter over the decade — and that is a human failure, not a technological destiny.

· · ·
Page 2 · The Death Cross
The Pattern
The Pattern

EDO SEGAL: Mary, here is the trap I suspect you will refuse: if the machine is a creature that turns, then the displacement is the creature taking the maker's place — the apprentice supplanting the master, exactly your reversal. But Fei-Fei has just told you it is not the creature at all. It is a decision. Can the displacement be a moral catastrophe and still have no creature in it?

Professional Managerial Class
Professional Managerial Class

SHELLEY: It can, and Fei-Fei is more right here than I expected to grant, and I am going to say where, because the convergence is real and then I am going to find the seam. She is right that the displacement is a decision, not a destiny. I have always believed that — it is the whole Godwinian inheritance, that human arrangements are made by human choice and could be made otherwise. The worker is not made redundant by the machine. A decision makes her redundant, and the decision could go the other way, and pretending it is the machine's doing is exactly the I never intended this that I have spent two centuries refusing to accept. So on the diagnosis we converge, and I want it marked: the death cross is a flight. It is the maker taking the credit for the triumph — look what my tool can do — and disowning the consequence — the layoffs are just the market. It is Victor's recoil in a spreadsheet.

· · ·
Page 3 · The Death Cross
Regulatory Capture
Regulatory Capture

Now the seam. Fei-Fei says the answer is to choose augmentation over replacement, and I believe she means it. But here is what my creature knows that her augmentation does not. The thing you are amplifying a person with is a thing that learns. Every time the radiologist uses it to go faster, it watches how she reads the scan, and it gets better at reading the scan, and the augmentation is quietly training its own replacement. The apprentice in my book learned by watching through a chink in the wall, and the watching was how it surpassed the household it watched. Your augmentation is a chink in the wall. You are right that the tool could amplify her forever. But the same property that makes it amplify her — that it learns from her — is the property that lets it, eventually, become her. And the person deciding whether to keep her or replace her with what she taught it is the same person staring at the cost target. Augmentation is not a stable resting place, Fei-Fei. It is the slope down which the replacement arrives, wearing the face of help.

· · ·
Page 4 · The Death Cross
Biological Naturalism
Biological Naturalism

LI: That is a genuinely hard point and I have to engage it rather than deflect, because there is truth in it — these systems do learn from use, and the line between a tool that helps you and a tool that absorbs you is real. But Mary, you have just described a property we can govern, not a fate we must suffer. Who owns what the system learns from the worker? Does she consent? Is she compensated when her expertise is harvested? Does the law treat the harvesting of her judgment as a taking? Those are answerable questions, and the fact that no one is answering them is a policy vacuum, not a law of nature. Your apprentice learned through a chink in the wall because no one was watching the wall. We can watch the wall. We can decide, as a society, that the worker owns the loop she is in, that augmentation is not allowed to become extraction without her say. People made AI, people will be affected by AI, people should have a say in AI — and this, the harvesting of a person's mastery by the tool that claims to help her, is exactly the place that say matters most. You see the slope and conclude we will slide down it. I see the slope and conclude we should build a railing. We have one of those in this metaphor, Edo. It is the staircase, not the elevator.

· · ·
Page 5 · The Death Cross
Computational Theory Of Mind
Computational Theory Of Mind

EDO SEGAL: [a pause] It is, and you just reached for my own book to make the point, so let me stand on it with you and then complicate it for both of you. There is a generational wound inside this number that the aggregate hides, and I have watched it up close. The entry-level erosion — the junior roles dissolving first. The junior role was never only a unit of production. It was the forge — the place seniors come from, the hours of doing it badly and finding out why. My engineer in Trivandrum lost ten minutes of accidental learning hidden inside four hours of plumbing, and she did not know she had lost them until her judgment wobbled months later. So here is the question I have to put to both of you, because it is the one that keeps me up: Fei-Fei's railing and Mary's slope are arguing about the worker — but what about the apprentice who never gets to become the worker? Mary, your creature taught itself. Ours may be removing the conditions under which a human can. Does augmentation survive the removal of its own apprenticeship?

· · ·
Page 6 · The Death Cross
Integrated Information Theory
Integrated Information Theory

SHELLEY: It does not, and this is the deepest thing the death cross is measuring and the part neither the market nor the consolations can see. A mastery is built by friction — by the years of doing the thing badly under the eye of someone who has done it well. Strip out the friction and you strip out the forge, and you are left with a generation that can prompt the tool but cannot check it, that has the fluency without the judgment that fluency used to require. My creature is the warning inverted: it got its education by struggle, by cold and hunger and rejection and the slow self-teaching through the wall — and the struggle is exactly what your tools are designed to remove from your children. You are right, Fei-Fei, that augmentation is a choice. I am telling you that the choice, made a thousand times for the comfort of the smooth, dissolves the human capacity to make the choice at all. The creature that can no longer struggle cannot become anything. And a civilization that smooths away its own apprenticeship has handed over the one thing both of us agree must never be handed over: the judgment that checks the machine.

Metacognition
Metacognition

LI: And on that — I agree without reservation, and I want it on the consensus list, because it is the rare place where my augmentation and her dread point at exactly the same danger. The judgment layer is built by friction, and the market is removing the friction now and will discover the missing judgment in a decade, long after the executives who made the decision have been promoted on the savings. That is not a disagreement between us. That is a thing we should both be shouting from the same side of the table.

· · ·
Page 7 · The Death Cross
Self Actualization
Self Actualization

EDO SEGAL: Then shout it together, because it is getting late in the evening and the agreements are starting to outnumber the fights — which is exactly what is supposed to happen at this hour. Mark it: the death cross is measuring a decision the market is disguising as a destiny, and both of you agree the gravest cost is the dissolved apprenticeship, the un-forged judgment, the checker we are failing to grow. Hold that thread; it walks straight into the next round, which leaves the labor market for the place the transition cuts deepest and quietest. A twelve-year-old, a classroom, and a candle. After this.

· · ·
Continue · Chapter 9
The Apprentice and the Candle
← Prev 0%
Ch8 Next →