The Death Cross and Who Owns the Data

EDO SEGAL: Let me set the table with numbers, because this round is about what the numbers mean. By early 2026 something like a trillion dollars of market value had drained out of the public software industry, while the model companies raised at valuations approaching a trillion each and the hyperscalers committed hundreds of billions in a single year of capital spending. I named the crossing the software death cross — the rung where the machine's capability crosses the human's and the arithmetic in the boardroom changes. I sat in those boardrooms. The question on the table, quarter after quarter, was: if five amplified people can do the work of a hundred, why pay for a hundred? Pedro, you've written that whoever has the best algorithms and the most data wins, and that the advantage compounds. So what is the death cross actually measuring — real capability, or a priced-in prophecy?

DOMINGOS: It's measuring both, and the dangerous part is the second one wearing the costume of the first. Let me give you the real mechanism, because it predates the hype and it's the thing I got right in 2015. Learning algorithms are seeds, data is the soil, and the learned model is what grows. Which means the organization with the most users gathers the most data, grows the best models, attracts more users, and gathers still more data — a network effect on data that compounds into dominance. That's not hype. That's structural, and it's why a handful of companies own this. So part of the trillion-dollar move is the market correctly pricing a genuine, compounding capability advantage held by very few hands.

But — and here's where Karl and I are about to agree violently — the other part is the market pricing a story. The story that the machine can replace the worker. Notice it doesn't have to be true for the death cross to happen. It only has to be believed long enough to clear a fiscal year. The executive doesn't need the model to do the junior lawyer's job. He needs the model to produce something that resembles the work product closely enough to justify cutting the role, and the quality erosion shows up later, diffusely, on customers and on the skeleton crew of seniors now babysitting the output. A mirage can absolutely restructure a labor market, Karl, because labor markets run on what decision-makers believe, and belief is the one thing hype manufactures cheaply.

POPPER: Now you are speaking my language exactly, and I want to name the specific intellectual crime, because it has a name and it is the one I hunted longest. It is historicism — the doctrine that the future is fixed and readable, that the river has a destination, and that the wise man's task is merely to align himself with the inevitable. Every prospectus raising that trillion dollars contains, somewhere, the sentence "this is inevitable." And the function of that sentence is not to describe the world. It is to disarm the reader. Because if the displacement is inevitable, then no one decided it, no one is responsible for it, and the worker thrown out of the forge was not fired by a person making a choice — she was simply standing in the path of the river. That is the great moral laundering of our age, and it is precisely the laundering I exposed in the historicists: the conversion of decisions into events and agents into bystanders. There is no such thing as the inevitable in human affairs. There are only choices, made by people, who then describe their choices as weather so that no one can vote on them.

DOMINGOS: And the proof that it's a choice and not weather is that you can watch the fork. I stood at exactly such a fork in my own research — you can build systems that compound the advantage into ever-fewer hands, or you can build for collective intelligence, aggregating and amplifying what a whole society knows. Those are different design choices, made by people, fundable or not. The technology doesn't determine which. What determines it is who owns the data and what they're optimizing, and right now that's a few firms optimizing for the compounding lock-in because that's what their capital requires. Karl's right that "inevitable" is a lie. I'll go further: it's a deliberate lie, because the people saying it stood at the same fork I did and chose, and "inevitable" is what you call your choice when you don't want to defend it.

POPPER: Prophecies are policies wearing a robe. And the robe is the most dangerous garment in politics, because you cannot argue with the weather. You can only argue with a man who admits he is choosing. The whole achievement of the open society — the entire fragile achievement — was to strip the robe off the rulers and force them to say we decided this, and you may decide otherwise. Your industry has reinvented the robe and stitched it out of trend lines.

EDO SEGAL: I want to route this through the kitchen table, because the aggregate trillion hides a particular wound and I've watched it up close. The entry-level erosion. Junior roles hollowing out first while senior roles hold — the floor dissolving before the ceiling. And every executive consoles the room with the old story: the tool makes things cheaper, demand explodes, more work for everyone, eventually. Maybe. But the junior role was never only a unit of production. It was the forge — the years of friction where the next senior gets made, where judgment is built by being wrong in small ways with small consequences. Pedro, your field is about to run an experiment on whether expertise can survive the removal of its own apprenticeship. Karl, you spent your life on how the critical faculty is built and how it atrophies. So, plainly, both of you: if we automate away the forge, where does the next generation of people who can check the machine come from?

DOMINGOS: Honestly? Nobody knows, and anyone who says they know is selling something. What I can say is the shape of it. The forge works on a delay — ten years of friction in, one senior out — and the market is removing the friction now and will discover the missing seniors in a decade, long after the executives who saved the money got promoted on the savings. Every firm is behaving rationally and the profession is being strip-mined of its future. And here's the part that should scare even me: the machine's quality ceiling is the quality of the human knowledge it learned from. Stop making seniors who can check it, and you stop making the very people whose judgment the next model needs to train on. The savant eats its own teachers.

POPPER: Then the death cross is measuring something neither capability nor prophecy fully captures. It is measuring the rate at which we are consuming the conditions of our own correction. The critical faculty — the disposition to doubt, to test, to find the black swan — is not innate. It is an achievement, built by friction, transmitted by apprenticeship, fragile across generations. A society can dismantle the forges in a decade and discover the loss in a generation, by which time it lacks the very people who could have noticed. That is the danger I would put above all others tonight, and notice that it has nothing to do with whether the machine is intelligent. A perfectly stupid machine, trusted past its competence, deployed to dissolve the apprenticeships, would produce it just as surely. Perhaps more surely, because no one would be watching for it.

EDO SEGAL: Mark this convergence and weigh it — it may be the heaviest of the night. You both locate the gravest danger not in the machine's power but in our dismantling of the human capacity to check it, and you both agree this danger is indifferent to the metaphysics, indifferent to whether anyone is home. Hold it close. Next round goes to the fear Pedro has fought hardest — the doom, the extinction story — and why the man who built the machine thinks the apocalypse is a category error. After this.