Heinz von Foerster

In the [YOU] on AI Field Guide

The cycle that began with [YOU] on AI asks what it would mean to see the machine clearly, without the narcotic of hype or the paralysis of fear. Von Foerster is the thinker in the cycle’s gallery who dismantles the most persistent confusion about what clear-sightedness would require. We keep asking whether AI is objective, whether it really understands, whether its outputs are trustworthy, as though these were questions about the machine alone. Von Foerster would smile and ask who is doing the looking. Every evaluation of a model is an observation made by an observer inside the system being observed, and the pretense that there is a view from nowhere—a neutral benchmark, an impartial audit, an objective standard against which the model’s behavior can be measured—is what he called the abrogation of responsibility dressed in the grammar of fact.

His trivial/non-trivial distinction cuts through the engineering reflex that treats AI alignment as a certification problem. The engineering culture of reliability presupposes that the thing under examination is, at bottom, trivial: that there is a rule, and that we can test behavior against the rule. A large language model is non-trivial in von Foerster’s exact sense: its output depends on its entire internal state, which was shaped by everything it has processed before, and the state space is so vast that exhaustive testing is impossible in principle. To demand the certainty that trivial-machine certification provides from a non-trivial-machine deployment is to demand something the architecture cannot supply. Von Foerster named the impossibility before the architecture existed.

And his ethical imperative cuts through the alignment debate with equal precision. The reigning question in responsible AI is: how do we align the system’s objectives with human values? Von Foerster’s question is different and, he would argue, prior: does the system, when it has run, leave the people it has acted on with more ways to act, or fewer? A system that maximizes a perfectly specified objective collapses the world toward that objective and forecloses the plurality it was supposed to serve. The imperative to increase choices is not an optimization target; it is a verdict to render on every optimization. A recommender system that learns a user and serves them more of what it predicts they will engage with is, by this measure, a choice-destroying engine, however well-aligned its recommendations are with the user’s expressed preferences.

Origin

Von Foerster was born in Vienna in 1911 into a family dense with engineers, architects, and artists—Ludwig Wittgenstein was a family connection—and trained in technical physics at the Technische Hochschule Wien and then at the University of Breslau. He arrived in the United States in 1949 carrying a manuscript about memory and a habit of mind that would define everything after: the refusal to describe a system while quietly standing outside it. The Macy Conferences on cybernetics, held in New York through the late 1940s and early 1950s, gathered Norbert Wiener, John von Neumann, Warren McCulloch, Margaret Mead, and a core of others to ask how feedback, control, and communication were the same problem in machines, brains, and societies. Von Foerster became the conference secretary and editor of its five published volumes, and he proposed the very name—cybernetics—that the field would carry. He was present at the creation, and he spent the rest of his life arguing that the creation had been incomplete.

The incompleteness was this: first-order cybernetics studied systems as if the cybernetician were a clean pane of glass. Von Foerster’s move was to fold the glass into the picture. In 1958 he founded the Biological Computer Laboratory at the University of Illinois, which he ran until the mid-1970s as a deliberately anti-hierarchical space for research on self-organization, neural modeling, and the computation of living systems. It was the BCL that sheltered Humberto Maturana and Francisco Varela while they developed autopoiesis, the theory of self-producing living organization that became the biological backbone of constructivism. Gordon Pask, Ross Ashby, and Margaret Mead all passed through the BCL’s orbit. It was an institution built to maximize the collision of perspectives—the ethical imperative turned into an organizational chart.

Von Foerster spent his last years in Pescadero, California, thinking and teaching to the end. He died in 2002, before any transformer model had run a single forward pass. He did not anticipate the machines. He anticipated the confusion we would have about them.

Key Ideas

Second-order cybernetics. First-order cybernetics studied observed systems; second-order cybernetics studies observing systems, including the observers themselves. The move is not merely reflexive; it has ethical consequences. Von Foerster’s formulation is deliberate: “objectivity is the delusion that observations could be made without an observer,” and to invoke objectivity is to abrogate responsibility. The implications for AI run deep: every claim that a model is objective, unbiased, or neutral is, on his account, a mechanism for laundering human choices through the grammar of fact, hiding who decided what the system should optimize for and how to weight the values in conflict.

Trivial and non-trivial machines. A trivial machine returns the same output to the same input every time; it is analytically transparent and certifiable. A non-trivial machine has internal states, and its response to any input depends on its entire history; it is in principle indeterminable. Large language models are non-trivial machines in this precise sense. You cannot exhaustively test a system whose relevant state space is astronomical and whose behavior shifts with history. The taxonomy predicts both the capabilities of the machine and the impossibility of the safety certification paradigm applied to it.

John von Neumann

Constructivism and eigenbehavior. Von Foerster held that the nervous system does not receive a ready-made world through its senses but computes one—generating stable regularities from undifferentiated signals and then operating inside those regularities as its world. He called the stable patterns that emerge from recursive self-application eigenforms: fixed points of the system’s own operations, which it then experiences as objects. A large language model builds a high-dimensional geometry of statistical relations and then operates inside that geometry as its world. Both the brain and the model are constructors, not mirrors. The difference von Foerster would insist on is that the brain’s construction belongs to a living system whose existence is at stake in getting the world workable, while the model’s construction belongs to no one.

The ethical imperative. “Act always so as to increase the number of choices.” Von Foerster was careful to say this was not a rule to be followed but a direction of travel—ethics, he argued, cannot be articulated as a code without becoming the abdication of responsibility it was meant to prevent. The imperative is deliberately not an optimization target; maximizing choice in a naive sense would be self-defeating. It functions as a verdict: after the system has run, do the people downstream have more meaningful options, or fewer? Applied to AI, it produces a systematic indictment of recommendation systems, engagement-maximizing feeds, and automated decision systems, which are, by construction, choice-narrowing engines. A writing assistant that completes your sentence before you have it is not increasing your choices; it is eliminating the one that mattered most.