Emergence, or the Order You Could Not Foresee

EDO SEGAL: There is a word doing enormous work in your field, Fei-Fei, and Ada has been circling it all night without saying it: emergence. The claim that capabilities appear at scale that nobody programmed and nobody predicted. Ada, I suspect you think it is a magic word — a place where the field says "and then a miracle occurs." Fei-Fei, I suspect you think it is the most important empirical fact of the era. So I am going to do something unusual and have you each define the other's strongest reading of it. Ada — give me emergence as Fei-Fei means it, at its most compelling. Then defend your skepticism.

LOVELACE: Very well, and I will be fair, because the fair version is genuinely unsettling. Emergence, as Fei-Fei means it: you build a network and a training procedure, you increase the scale — more data, more parameters, more computation — and at certain thresholds the system does not merely get better at what it could already do. It does new things. It acquires a capability that was not present at smaller scale and that no one wrote a line of code to produce — it begins to translate, or to reason in steps, or to see in a way it could not see before — and these capabilities surprise the very people who built the system, including people, like Fei-Fei, who had been predicting for years that networks would surprise us. The structure was not in the design. It condensed out of the scale. And the honest unsettling part, which I will not hide behind, is that "it was implicit in the data" is cold comfort when no human could have extracted it, predicted it, or even recognized it as possible beforehand. That is her case, and it is strong.

Now my skepticism, and it is not the cheap kind. I do not doubt the phenomenon — the capability appears, the people are surprised, I grant all of it. I doubt the inference you draw from the surprise. Surprise is a fact about the expectations of the observer. It is not a fact about the machine. When the capability "emerges," nothing miraculous happened in the engine; an enormous deterministic computation produced an output that your prior model of the computation did not lead you to expect. The emergence is in the gap between the system and your understanding of it — and that gap is a fact about the poverty of our understanding, not about the richness of the engine's being. You have built something whose behavior you cannot predict, and you have named your own surprise "emergence" and located it in the machine. It is the loom again, Fei-Fei. You are finding in the engine — now in its behavior rather than its output — a property that actually lives in the relation between the engine and your limited sight of it.

LI: That is the most serious version of the skeptical case I have heard, and I want to honor it by conceding the half that is true and then showing you why the other half does not hold. The true half: yes, "emergence" is partly a statement about our ignorance, and my field is sloppy about this, treating surprise as if it were evidence of metaphysical novelty. Guilty. Some "emergent" capabilities, when measured carefully, turn out to be artifacts of how we drew the graph. Ada is right to make us pay for the word.

But here is what her account cannot reach, and it is the thing I actually lived. When I built ImageNet, I was not surprised in the way Ada describes — I was not a naive observer with a poor model who got startled. I had spent years predicting that scale would produce structure. I designed the experiment expecting the surprise. And the surprise came anyway, in a form I had specifically braced for and still could not have specified in advance. That is a different epistemic situation from the man startled by the avalanche. It is the situation of the scientist who predicts that an experiment will reveal something she does not yet know — and is then taught, by the experiment, a thing she could not have deduced from her own mind. Ada, when an experiment teaches its designer something the designer could not have derived without running it, we have a name for that. We call it discovery. Not because the observer was ignorant — every scientist is ignorant before the experiment, that is the point of experiments — but because the world, through the apparatus, supplied a truth the mind did not contain. You want to relocate all of it into "the poverty of our understanding." But the poverty of our understanding relative to a process that can repair it is exactly the condition under which every real discovery in history has occurred. You have just described science and called it an illusion.

LOVELACE: No — you have just described measurement, and conflated it with origination, and the conflation is the oldest error in the philosophy of the instrument. When the telescope showed Galileo the moons of Jupiter, the telescope discovered nothing. It revealed. The moons were there; the instrument made them visible to a mind that then did the discovering — formed the belief, grasped the consequence, was changed. Your network is a magnificent telescope pointed at the structure latent in fifteen million images. It reveals structure to you that your unaided eye could not reach. I grant it fully — it is a glorious instrument and it has extended human sight the way the telescope did. But the telescope did not discover the moons. Galileo did, by looking through it and understanding. Your network reveals the structure. You discover it, when you look through the network and are changed. Do not promote the telescope to the astronomer because the astronomer could not have seen the moons without it. The whole history of instruments is humans seeing further through things that see nothing.

EDO SEGAL: That is the sharpest exchange of the night, and I want to name what the reader just watched, because it is the seam of the whole evening laid bare. Fei-Fei says: an apparatus that teaches its designer a truth she could not derive is doing discovery — that is just what discovery has always been. Ada says: no, the apparatus reveals, and the discovery happens in the mind that looks through it and is changed; promoting the instrument to the discoverer is the error. And notice — this is the thing — neither of you disputes a single fact about the machine. You agree on exactly what it does. You disagree about where, in the chain from the apparatus to the changed mind, the word "discovery" attaches. Ada, you said an hour ago this was a question about us. It is. Hold it. We break, and when we come back I want to follow the river all the way down to the child — because Fei-Fei's whole case begins with how a child learns to see, and I want to know what the machine drank that the child also drinks, and what it can never drink at all.