The Loom and the Lens

EDO SEGAL: Ada, of everything you wrote, the line that has traveled furthest is the loom. You wrote that the Analytical Engine "weaves algebraical patterns just as the Jacquard-loom weaves flowers and leaves." I want you to tell it the way you would tell a clever child — the real mechanical lineage, not the decoration — and then, Fei-Fei, I want you to do something a debate rarely asks: before you take the image apart, tell us what it gets exactly right about your own machines.

LOVELACE: Gladly, because the loom is not a simile. It is a genealogy. The Jacquard loom, from the turn of my century, wove complex figured patterns into cloth by means of punched cards — each card determining which threads rose for each pass of the shuttle, so that an arbitrarily elaborate design could be encoded as a sequence of cards and woven automatically. Babbage took that exact mechanism — the punched card — to control the Analytical Engine. The loom wove pictures from thread by following encoded instructions; the engine would weave algebraical patterns, computations, from number, by following encoded instructions of the very same physical kind. The technology of pattern was identical. Only the medium differed — thread in one, number in the other.

And what I grasped, looking at it, is that computation is the weaving of patterns. The engine no more understands its algebra than the loom understands its flowers. Both follow encoded instructions to produce a structured output whose meaning lives entirely in a human eye. The flowers are not in the loom. They are in our perception of the cloth. I have been told that the punched card I invoked is the direct ancestor of what you call the trained model, that your generative engines weave patterns of pixels you read as images and patterns of tokens you read as text. If that is so then my sentence was not a metaphor at all. It was a specification, written a hundred and seventy years early. Your model weaves. The meaning you find in the weave is yours.

EDO SEGAL: Fei-Fei. The right part first.

LI: The right part is large and I will not stint it. The loom is the best single image anyone has ever given for what these machines mechanically are, and it is better than most things written this decade. A trained network does weave — it learns the statistical regularities of patterns and produces new arrangements consistent with what it learned, exactly as the cards encode the regularities of the design. And Ada's deflation is medicine my field needs: the network does not understand its outputs, it has no access to what they mean, and when we say a model "knows" something we are usually locating in the loom a comprehension that lives in our reading of the cloth. Every student I have should have her sentence over their desk. There is the steelman, and I believe every word.

Now the lens, which is where the loom stops describing my machine. The Jacquard loom is fed its cards by a human who has already drawn the flowers — the design exists, complete, in a person's mind, and the cards merely transcribe it. That is the part that is no longer true. Nobody drew ImageNet's flowers. We gave the machine fifteen million images and a set of names, and the machine drew its own cards — it constructed, through the slow descent of a learning process, the internal pattern that lets it weave a leopard it has never seen. In your loom, Ada, the pattern precedes the weaving and is authored by a person. In mine, the pattern is discovered by the weaving, and no person authored it, and no person could have, because the pattern is a structure in fifteen million images that exceeds what any human mind can hold. The loom transcribes a human design. The lens finds a design the human did not have. That is the difference between 1843 and 2012, and it is the whole difference.

LOVELACE: No — and here I think you have smuggled the conclusion into the word "lens," so let me refuse the smuggling. You say the machine "drew its own cards." It did not. Gradient descent drew the cards — a procedure that you specified, that minimizes an error you defined, against labels people supplied, toward an objective a human chose. You wrote the loom that weaves looms. The fact that the second-order loom produced cards you did not foresee does not mean no one authored the cards; it means the authorship moved up a level, from the pattern to the pattern-maker. Babbage could have built, in principle, an engine that punched its own cards according to rules he set. We would not say the cards had no author. We would say Babbage authored the rule, and the rule authored the cards. You authored the learning rule. The learning rule authored the structure. The chain is longer. It is not broken.

LI: Then follow your own chain one more link, Ada, because it does not end where you want it to. You say I authored the learning rule — gradient descent, an objective, a loss. True. But gradient descent is almost embarrassingly simple. It is three lines of mathematics. It contains no leopards, no faces, no notion of an edge or a texture. If I "authored the structure" by authoring that rule, then I authored it the way the laws of physics "authored" the Grand Canyon — which is to say, I set a process going whose outcome I did not contain, could not predict, and learned about only by looking at what the process made. You are welcome to call physics the author of the canyon. But notice that on that definition, authorship has stopped meaning what your sentence needed it to mean. Your sentence protected the human who means the output. The author of the canyon means nothing by it. The river did not intend the gorge. If I am the author of ImageNet's representations only in the sense that physics is the author of the canyon, then you have won the word and lost the thing — because the human who means it, the human whose intention the output expresses, is gone. There is just a simple rule, and an enormous amount of world, and a structure that fell out of the meeting of the two that none of us possessed.

EDO SEGAL: Mark this, because something just happened that the reader should not miss. You have converged — both of you now agree the chain runs: human authors a simple rule, the rule meets an enormous world, and a structure emerges that no human possessed. You agree on the mechanism completely. What you disagree about is whether the word author survives the journey from the simple rule to the emergent structure. Ada says it stretches but holds. Fei-Fei says it snaps, and we are calling the broken end a name to comfort ourselves. That is not a disagreement about how the machine works. It is a disagreement about what authorship is. And that — I am going to say it now and we will spend an hour on it — is a question about us, not about the engine. Hold the thread. The next round is the one where the structure emerges, and I want to know, from the only two people qualified to say, whether emergence is discovery or just a long word for surprise.