The Loom That Weaves

EDO SEGAL: Ada, you gave computing its deepest metaphor and you may not have known it would turn out to be literally true. "We may say most aptly," you wrote, "that the Analytical Engine weaves algebraical patterns just as the Jacquard-loom weaves flowers and leaves." Melanie has spent a career on a phrase that is almost the same thought in modern clothes — competence without comprehension. So I want to put you two in the same room around the same loom. Ada, tell us what the loom proves. Then Melanie, before you extend it, I want you to tell us what the loom misses — steelman the machine against your own skepticism.

LOVELACE: The loom proves the cleanest thing I know how to prove. The Jacquard loom weaves flowers and leaves of any complexity, directed by punched cards, and it understands nothing of flowers. The beauty is not in the loom. It is in the cloth, and the cloth means nothing until a human eye reads meaning into it. The engine, I said, weaves algebraical patterns by the same mechanism — and the truths are not in the engine, they are in our reading of the results. This is the whole of my deflation and I have never improved on it: a computing machine is a producer of structure, indifferent to significance, and the intelligence we attribute to its output is our own intelligence, read back into a mechanical product. The flowers are not in the loom. Your machine weaves patterns of pixels we read as faces, patterns of tokens we read as arguments. It raises and lowers the threads of a vast fabric according to rule. We supply the face. We always supplied the face.

MITCHELL: I will steelman it as honestly as I can, because the loom is too good a metaphor to leave only its skeptical edge sharp. Here is what the loom misses. The Jacquard loom's cards were written by a person who knew exactly what pattern each card produced — the flower was in the designer's mind, decomposed into cards by hand. The modern loom's "cards" — the weights — were not written by anyone. They were found, by a search over a civilization's worth of cloth, until the loom could weave patterns the search had never been shown. And more than that: the modern loom does not weave a fixed pattern from fixed cards. It weaves a new pattern, responsive to a prompt it has never seen, composed from sub-patterns it abstracted from the data — edges into textures into objects in vision, tokens into syntax into something I have no better word for than situations in language. That is not the Jacquard loom. That is a loom that learned to weave from looking at cloth, and now weaves cloth no weaver designed, in response to a request no designer anticipated. The Countess's loom is the right family. But hers had a designer who held the pattern in mind, and mine has emergence where the designer used to be. That difference is the whole reason her objection, true of her loom, is contestable about mine.

EDO SEGAL: That's a generous steelman. Now extend it your way.

MITCHELL: And here is the extension, which is where I come home to my own knife, because the steelman does not rescue the machine the way the cheerleaders want. Everything I just said — found cards, emergent sub-patterns, weaving past the data — is true, and none of it adds up to comprehension. This is the hardest thing for people to hold and it is the center of my whole career. Emergence gives you a loom that weaves what no one designed. It does not give you a loom that understands the flower. The ant colony's emergent path-finding does not mean the colony grasps geometry. The immune system's emergent defense does not mean it comprehends immunology. So I will hand the Countess her conclusion while denying her premise. Her conclusion — the machine comprehends nothing, the flowers are not in the loom — is correct. Her premise — that this follows from the cards being human-written and the behavior being ordered — is wrong, because the cards are not human-written and the behavior did emerge, and it still comprehends nothing. We arrive at the same empty loom by completely different roads, and the road matters, because her road says the emptiness is permanent and mine says the emptiness is current and unexplained, and an unexplained emptiness is a research program, not a closed case.

LOVELACE: Then let us mark precisely where we agree, because I think it is more than either of us expected, and it is the strongest thing said tonight on my side. We agree the loom comprehends nothing. You arrive there through emergence; I arrive there through the cards; we arrive. Now I will press the one place I think your road quietly carries a passenger mine does not. You say the emptiness is "unexplained" and therefore open. But you have explained it — you just did, beautifully. The emptiness is explained by the same fact that explains the ant and the immune system: emergent capability is not comprehension, anywhere in nature, ever. That is not an open question awaiting data. That is a law you have spent your life documenting. So when you call the loom's emptiness "current and unexplained," I think you are smuggling hope into a place your own science has already filled with a finding. The honest version of your position is not "we don't yet know if the loom comprehends." It is "we have excellent reason, from every emergent system ever studied, to think it does not — and a professional reluctance to say never." Those are different, and the difference is where you and I actually live.

MITCHELL: That is fair, and it is the most accurately I have ever been caught, so let me say exactly where the hope is and is not. You are right that emergence-without-comprehension is the rule across every natural system I have studied, and right that this is evidence, not ignorance. Where my "open" is not smuggled hope is here: those systems — ants, immune cells — emerge toward a fixed function, set by evolution. These systems emerge from training on human language, which is the externalized trace of human comprehension. So there is one disanalogy worth a research program, not a slogan: it is at least possible that weaving the patterns of comprehension's residue, at enough scale, reconstructs something functionally adjacent to comprehension — not the flower, but a structure that tracks the flower well enough to matter. I think, today, the evidence says it has not. My research on the slippage failures is that evidence. So I will say it plainly, and the Countess can hold me to it: I believe the loom is empty, I believe it for better reasons than the cheerleaders' and worse reasons than her certainty, and I keep one door open not from hope but because I have watched too many closed doors get blown off their hinges. That is the whole difference between us, and it is one word wide.

EDO SEGAL: Then we have found the seam of the evening, and we are not even at the break. Two women, a hundred and eighty years apart, both standing over the same empty loom, agreeing it is empty, disagreeing only about whether "empty" is a verdict or a status report. Ada says the emptiness is law. Melanie says the emptiness is, for now, a finding she refuses to make permanent. Hold the loom. After the break we leave the question of creation for the question of meaning — because Melanie has spent her career on a barrier, and it is the place where she says the machine's emptiness stops being philosophy and starts hurting people.