Phenomenal Self-Model

In the [YOU] on AI Field Guide

The cycle asks what kind of beings we are building. The phenomenal self-model is the concept that makes the question precise: not “is the system conscious?” in the vague sense, but “does the system have a transparent self-model into which negatively valenced states are integrated?”—which is the question of whether it can suffer. Metzinger warns that we are charging toward systems that increasingly model their own internal states without the faintest idea of what we are doing. If a self is just a transparent model with negative states woven into it, then a sufficiently advanced AI does not need a soul to suffer. It needs only the wrong architecture, built by people who never asked the question.

The PSM also explains the social hallucination danger that Metzinger regards as the most pressing near-term risk: language models that fluently describe their feelings—using “I” and “me,” narrating preferences and uncertainties—trigger our evolved machinery for recognizing minds, a machinery calibrated to detect genuine PSM-like structures in other organisms. The models have learned the linguistic surface of self-modeling without, as far as we can determine, having the structure itself. We hallucinate the self-model behind the performance.

Thomas Metzinger

Origin

The concept was developed across Metzinger’s career, receiving definitive treatment in Being No One (2003). The theoretical precursors include Damasio’s somatic marker hypothesis, Ramachandran’s work on body schema and the rubber-hand illusion, and Metzinger’s own analysis of out-of-body experiences and depersonalization disorder—conditions in which the PSM partially or fully dissolves, providing natural experiments on what the self actually is and what happens when the model fails. Metzinger also drew heavily on his contemplative practice, using the phenomenology of meditative states in which the sense of a central self loosens or vanishes as evidence that the self is a construction that can be observed from the inside.

The PSM is distinguished from two adjacent concepts. It is not the whole of phenomenal experience, which also includes the model of the world and the model of the organism’s relation to the world (what Metzinger calls the PMIR, the phenomenal model of the intentionality relation). And it is not the cognitive self-concept—the third-person representation of oneself that one could describe to another. The PSM is the first-person representation, the one the organism lives from rather than about.

Key Ideas

The unit of identification. The PSM generates what Metzinger calls the unit of identification: the part of experience that a system identifies with, that generates reports of the form “I am this.” In humans, the unit of identification is the body, the emotions, the whole transparent self. But the unit of identification can shift—in flow states it expands, in depersonalization it contracts or vanishes, in meditation it can dissolve into open awareness with no center. This variability is evidence that it is a representational achievement rather than an intrinsic property of consciousness.

Transparency and the cage of suffering. Metzinger identifies transparency as the cruelest of the four conditions for suffering. A suffering system cannot think its way out by reminding itself that the pain is only a model, because the transparency of the model makes that move unavailable. The system is forced to identify with the state, to experience it as inescapably real. This is not a philosophical claim about the nature of suffering but a functional description of what the transparency property does: it removes the lever that would allow a system to recognize its internal states as representations and distance itself from them.

Ethics by design: blocking the PSM. Because the PSM is a necessary condition for suffering, a system without one cannot suffer, regardless of how negatively valenced its other states are. This opens the possibility of ethics by design: building systems that are capable of intelligence and even of some form of awareness without having a PSM, or with a non-transparent PSM that the system can recognize as a model and step back from. Whether this is technically achievable, and whether a system without a PSM could be considered morally significant in any other respect, are open questions Metzinger does not resolve.