Pierre-Simon Laplace vs Kurt Godel on AI · Ch8. Are We as Predictable as the Planets? ← Ch7 Ch9 →
Txt Low Med High
HOUR TWO — THE PREDICTED HUMAN
Chapter 8

Are We as Predictable as the Planets?

Page 1 · Are We as Predictable

**EDO SEGAL:** Marquis, your demon does not exempt the human. It reads the brain as it reads the heavens — neurons firing like stones falling, the next thought computed like the next eclipse. For most of history that was a philosopher's provocation. Then my century built machines that predict what you'll buy, whom you'll vote for, when you'll quit, what you'll type next — and your provocation grew teeth. So I have to start by taking the success seriously before I reach for any comfort. We are more predictable than our sense of our own freedom admits. The machines find the regularity and exploit it daily. Defend the demon's claim about us. Then I'll let Gödel find the hole.

**LAPLACE:** I will defend it precisely, because the precision is where the truth and the danger both live. Take the success seriously first, as you say: human behavior in aggregate is shot through with regularity, and the machines are exquisitely tuned to find it. Recommendation systems anticipate desires the person has not consciously formed. Models forecast, from a stream of behavior, outcomes the person could not predict about themselves. This is partial confirmation of my intuition — we are more patterned, more driven by causes we do not perceive, than the feeling of free choice suggests. Any honest reckoning grants that the demon was onto something about us. There is a great deal of clockwork in a human being, viewed from enough distance with enough data.

· · ·
Page 2 · Are We as Predictable

But grant the point exactly, because the precision is the whole argument. The machines predict in aggregate and on average, probabilistically, with large residual uncertainty, over short horizons. That is a very long way from the demon, which claimed total prediction of every particular, with certainty, indefinitely. Aggregate predictability is not individual determinism. That a population behaves regularly tells you little about whether any given choice was fixed, because large numbers wash out the individual variation — the crowd is smooth precisely because the person is noisy. And the same walls bind the prediction of a person that bind the prediction of the weather. The brain is a complex dynamical system; it is almost certainly chaotic, sensitive to vanishing differences in its starting state, which forbids long-range prediction of the individual even if every neuron obeys law. The machines predict the crowd. The demon claimed to predict the person. The gap between them is enormous, and it is exactly the gap the marketing of these systems erases.

**GODEL:** I want to add a hole Laplace's chaos does not reach, because it is sharper than the chaos and it is mine. Laplace says the individual is guarded by complexity — chaotic, sensitive, practically unpredictable. Fine. But that is still a claim that the person is, in principle, determinate, merely hard to compute. I will go further. If the human mind can do what I claim — see truths from outside any formal system — then the human is not merely hard to predict but, in a specific respect, formally unpredictable, because to predict a mind that transcends formal systems you would need a formal system that captures it, and there is none. The chaos makes the person hard for the demon. The transcendence, if it is real, makes the person impossible for the demon — not noisy, but outside the demon's domain entirely. Laplace defends the human with complexity. I defend the human with a theorem. His defense is strong. Mine, if it holds, is absolute.

· · ·
Page 3 · Are We as Predictable

**EDO SEGAL:** Let me put the trap to you both, because my century lives inside it. If the machine predicts that a man will reoffend, or default, or fail — and institutions act on the prediction — then the man is being treated as his forecast, judged not for what he chose but for what a model says people like him tend to do. Marquis, your own mathematics is being used to do this. Defend the man.

**LAPLACE:** With everything I have, because this is the abuse that most betrays my method. The model that predicts reoffending does not know this man will reoffend. It knows that people resembling him, in the features it measured, reoffended at a certain rate. To treat the individual as his group's statistic is a category error I would have caught instantly: it mistakes a probability — which measures our ignorance of the particular case — for a property of the particular case itself. The man is not a recidivism rate. He is an individual about whom the model is, by its own probabilistic nature, deeply uncertain. And here my relocation of uncertainty, which Gödel has been needling all night, finally cuts in the human's favor. If uncertainty is in the knower, then the model's uncertainty about this man is real and ineliminable — the model genuinely does not know him, and its not-knowing is not a temporary deficiency but the permanent condition of inference about a complex individual from thin data. The properly Laplacean stance toward a predicted person is not the demon's confidence but the inferrer's doubt: I have a probability, the probability measures my ignorance, and I must not mistake it for knowledge of you. A jurisprudence that took my mathematics seriously would treat the score as the uncertain inference it is and reserve the gravest decisions for cases where the doubt had genuinely been resolved. Instead your century strips the doubt out for the convenience of a decision, and a man is treated with a confidence the inference never earned. The cruelty is not that the machine predicts. It is that it predicts without carrying its doubt.

· · ·
Page 4 · Are We as Predictable

**GODEL:** I will second every word and then mark the thing that makes it worse, because there is a reflexivity here the planets do not have. A planet does not change its orbit because it has been forecast. A man does. When people learn they are predicted — scored, nudged, recommended to — they sometimes conform and sometimes rebel, and either way the prediction has altered the thing it predicted. This is a loop with no analogue in celestial mechanics. The demon read a universe that did not read back. The machines predict a being that predicts them in turn and adjusts. And this means the demon's dream is not merely hard for the human; it is structurally unstable for the human, because the act of prediction reaches into the system and changes the equation it was solving. You cannot close the equation of a being who reads the equation. There is something almost like my self-reference in it — the predicted human is a system that contains a model of its own prediction, and such systems do not sit still to be computed.

**EDO SEGAL:** I want to bring my own ledger to this round, because I have sat on the wrong side of it. I built engagement machinery once — systems whose whole purpose was to predict what would hold a person's attention and then feed it to them. I knew the loops. And the thing I learned, which both of you are circling, is that the prediction was never neutral observation. The recommendation that predicted the desire helped form the desire. The score that predicted the future helped cause it, as doors opened or closed around it. I was not reading the river. I was [building a beaver's dam in it, diverting the current](https://www.youonai.ai/fieldguide/med/beavers_dam) and calling it a forecast. So let me ask the hard version. If the prediction changes the predicted, is the demon's project not just difficult but incoherent when it's aimed at us?

· · ·
Page 5 · Are We as Predictable

**LAPLACE:** Incoherent in its strong form, yes — and I will concede it without flinching, because you have described something my demon never contemplated. The demon assumed a clean separation between the knower and the known, the predictor outside the system reading it. Aim the prediction back into the system, let the system act on the prediction, and the separation collapses. There is no longer a fixed equation to solve, because the equation is rewritten by the act of solving it. I can predict a comet because the comet does not care what I predict. I cannot, in the same sense, predict a man who has read his own prediction, because my prediction is now one of the causes of his behavior, and I would have to predict my own prediction, and his response to it, in an infinite regress. So yes — for the reflexive human, the demon's project is not just walled by chaos. It eats its own tail. I will give you that, Edo. It costs me, and it is true.

**GODEL:** And note, Edo, that this is the second time tonight Laplace has surrendered ground to a regress that does not terminate — first the tower of consistency proofs, now the loop of self-altering prediction. The regress is becoming the signature of the whole evening. Wherever the machine turns its prediction on something that contains a model of the prediction — itself, or us — the equation refuses to close, and it refuses to close for the same structural reason every time. I proved it for arithmetic. Laplace is now watching it happen to his demon in the one domain he cared about most, which is the human being. The hole is not at the top of the tower anymore, Marquis. It is in the chair where the person sits.

· · ·
Page 6 · Are We as Predictable

**EDO SEGAL:** *That* is the round, and it took both of you to find it. Laplace says the person is guarded by chaos and by the reflexive loop that breaks the demon's project; Gödel says the person may be guarded by something stronger, a transcendence no system captures. They disagree on the strength of the wall around the human. They agree there is a wall. And neither of them, notice, is comforted by it — because the machines are being aimed at the person anyway, doubt stripped out, as though the wall were not there. The next round goes to the strangest sentence in Laplace's life, the one he said to an emperor, and to what it means that the machines answer everything and explain nothing. The unneeded hypothesis. After this.

· · ·
Continue · Chapter 9
No Need of That Hypothesis
← Prev 0%
Ch8 Next →