Is Induction a Myth?

EDO SEGAL: I want to open this round with a confession instead of a question, because my best questions come out of wounds. I started in Assembler. I was raised by the machine code, and for fifty years every tool I used demanded that I learn its language, pay a tax to convert my intention into its grammar. In the winter of 2025 I watched that tax go to zero. I stood in a room in Trivandrum with twenty engineers and watched each one become capable of more than all of them together, in a week, because for the first time the machine met them in their tongue. I wrote that this was the great inversion. But Karl, here is the part that should trouble me and didn't until I read you closely: every one of those engineers was trusting a machine that learned the way you say no one can learn. By induction, from the pile. So take the scandal apart for me, slowly, and tell me whether my Trivandrum week rests on a myth.

POPPER: Your week is real, and it rests on a confusion that is older than computing, so do not feel singled out. Let me be precise about what the scandal is and is not. The scandal is not that generalizing from experience is useless — obviously it is useful; we could not cross a street without it. The scandal is that it is not justified, and that the difference between useful and justified is the difference between living and knowing. When your engineers trusted the machine, they were doing what all animals do — extrapolating a regularity — and most of the time the regularity held, because the world is, locally, fairly stable. The amoeba does this. The amoeba is an inductivist. And the amoeba dies when the regularity breaks, having no idea why, because it never had a theory, only a habit that worked until it didn't.

What I argued is that human knowledge — science — is precisely the thing that escapes the amoeba's fate, and that it escapes not by generalizing better but by generalizing boldly and then attacking the generalization. Einstein did not arrive at relativity by observing many fallings and averaging them. He conjectured a structure for spacetime — a wild guess, against the data of common sense — and then specified, in advance, the observation that would destroy it. The light bends by this much or I am wrong. That is the move the amoeba cannot make and the move I do not yet see in the machine. The machine generalizes like the amoeba, at superhuman scale, with no theory it could expose to a decisive test. So is induction a myth? The justification of induction is a myth — there is no such justification, there never was, and a machine built on the pretense that there is will inherit the amoeba's blind spot exactly where it matters most: at the edge, in the new situation, on the morning the sun does something it has never done.

DOMINGOS: I want to grant the amoeba and then take it away from you, because I think the amoeba is smarter than your story allows and the machine is smarter than the amoeba. Here's the move you're not crediting. When a learning algorithm fits data, it isn't averaging fallings. It's searching an enormous space of possible explanations and picking the one that predicts held-out data while staying simple — and "staying simple" is doing exactly the work you want. The bias toward simplicity is a bias toward the explanation with the fewest ways to be accidentally right, which is to say the most falsifiable one, which is to say your one, Karl. We even have a theorem that should make you happy and miserable at once: the no-free-lunch result, which says that no learner can generalize at all without prior assumptions about the world. Pure induction from data alone is mathematically impossible. You cannot learn from examples unless you bring a conjecture — a bias — to the table first. So my field proved your point with equations. Learning is conjecture plus data, never data alone. The machine is not the amoeba. The amoeba had one fixed habit. The machine carries a vast space of conjectures and lets the data shoot them down.

POPPER: [ Now I am genuinely delighted, and I want the audience to see why. ] Pedro has just told me that his own field has a proof that induction-from-data-alone is impossible, and that all learning requires a prior conjecture the data then tests. That is my philosophy. That is exactly my philosophy, rendered as a theorem. I could not have asked for a more handsome vindication, and I accept it with both hands. But watch what it does to your position, my friend, because it cuts toward me. If the machine cannot learn without a prior bias — a conjecture it brings before the data — then the crucial question is no longer "did it see enough examples." The crucial question is: what conjecture did it bring, and is that conjecture exposed to refutation, or is it hidden in the architecture where no one can attack it? You have moved the whole game onto my ground. The bias is the theory. And a theory you cannot see is a theory you cannot refute.

DOMINGOS: And here's where I stop agreeing, because you've smuggled in "cannot see." The bias isn't hidden mysticism. In a neural network the bias is the architecture and the simplicity pressure, and we can study it, ablate it, change it, and watch what happens. It's not a sermon, it's a knob. You keep wanting the conjecture to be a sentence a philosopher can inspect. Sometimes the most powerful conjecture about the world isn't a sentence — it's a shape, a structure, a geometry of similarity that no sentence captures. Your physics envy is showing, Karl. Not every refutable theory wears the costume of Einstein's clean prediction. Some of them are distributed across a million parameters, and they get refuted the same way Einstein's did — by failing to predict — just not in a single dramatic eclipse.

EDO SEGAL: Before I close this, I want to seat a ghost at the table, because he's been hovering since the first sentence. Thomas Kuhn. He argued that science doesn't actually proceed the clean Popperian way — that working scientists cling to a paradigm, ignore the anomalies, patch the theory rather than kill it, until a crisis forces a wholesale revolution. Karl, you and Kuhn fought this out for decades. And here's why it's not a museum piece: the machine is the ultimate paradigm-clinger. It will defend the consensus in its training data against every anomaly, because the consensus is what it was rewarded for predicting. So whose picture of science does the machine confirm — yours or Kuhn's?

POPPER: Kuhn described the pathology and mistook it for the method. Yes — scientists cling, patch, defend their paradigm past the point of honesty. I never denied that they do this; I denied that they should, and that the clinging is what makes it science. What makes it science is the small, stubborn minority who turn on the paradigm and try to break it — and the institutions that protect them from the majority who would rather not be disturbed. Kuhn's revolutions happen because somebody refused to patch and insisted on the refutation. So when you tell me the machine is a paradigm-clinger, you are telling me the machine has the vice of normal science with none of the virtue that redeems it. It will defend the consensus and it will never be the heretic. It is Kuhn's worst scientist, scaled, with no Popperian rebel anywhere inside it. That is not a neutral fact about the machine. It is the precise reason it cannot, on its own, advance knowledge — it can only consolidate it.

DOMINGOS: I'll grant the machine is a clinger and add the engineering reason, because it sharpens Karl's point into something actionable. A model trained to minimize prediction error on a fixed corpus is, mathematically, a consensus-seeker — it converges on the most-supported answer, which is exactly the paradigm. The heretic move — betting against the corpus — looks identical, to the loss function, to a mistake, and gets trained away. So if you want the rebel, you can't get it from prediction alone; you have to add something that rewards productive disagreement with the data, and nobody's commercial incentive points there. Karl's right that the machine is structurally conservative. Where I'd push Kuhn back is that the conservatism isn't a flaw to mourn — it's a tool to use. You want a consensus engine and a heretic, working together, the way real science is a community of clingers and a few rebels. We've built the perfect clinger. We just haven't built the rebel, and we're pretending the clinger is the whole of science.

EDO SEGAL: Let me restate what I think just happened, because the reader can't see your faces and I want this marked. Karl, you came in saying induction is a myth and the machine is built on it. Pedro answered that his field proved pure induction impossible, that all learning is conjecture-plus-test — and you said, wonderful, then the only question is whether the conjecture can be attacked. And Pedro said it can, by failing to predict, even when it's a shape and not a sentence. So you've converged on something startling: you both agree the machine learns by conjecture and refutation. You disagree about whether a conjecture you cannot read can be genuinely refuted, or only watched. Pedro — is a theory you can't read a theory at all?

DOMINGOS: It's a theory you can't explain, which is a real cost, and I won't pretend otherwise — explanation is one of the five tribes' crown jewels and the connectionists threw it away. But "can't read" and "can't refute" are different claims, and Karl needs them to be the same claim, and they're not. I refute an unreadable model every day. I show it a case from outside its comfort and it falls on its face and I have refuted it, cleanly, without reading a single weight. The autopsy is hard. The death is obvious.

POPPER: Then we agree the death is the thing, and I will hold you to it for three hours, because everything turns on whether anyone is staging the death honestly or merely admiring the life. But I will give you this, Pedro, and I give it gladly: I came expecting an inductivist and I have found a falsificationist who does not know the word. The machine you describe — conjecture, simplicity, severe test against the unseen — is not the enemy I prepared to fight. The enemy I prepared to fight is what the world has made of your machine: an oracle that predicts and is therefore believed, with the testing quietly dropped because the predicting is so impressive. The corruption is not in your mathematics. It is in what happens when your mathematics meets a marketplace that wants certainty and will pay for the costume of it.

DOMINGOS: On that — the marketplace dropping the test because the demo is shiny — we agree completely, and I'll say it louder than you when we get to the money. The hype is the enemy of the science. Always has been.

EDO SEGAL: Mark that — the first convergence of the night, and it is a big one: the danger is not the learning, it is the dropping of the test once the learning impresses. Hold it; we will need it on a higher floor. The next round goes to the place where the dropped test does its worst damage — the moment the machine confabulates with a straight face, and what Karl's word pseudoscience has to say about a system that can explain anything. After this.