The Beholder's Share

In the [YOU] on AI Field Guide

The cycle that began with [YOU] on AI takes the reliability of perception as one of the casualties of the AI transition: a moment in which the category of “real” is being actively destabilized by technologies that can manufacture its perceptual signatures. The beholder’s share is the biological explanation for why this destabilization is not merely a technical puzzle but a profound challenge to the epistemic habits humanity has developed across its entire history. For most of human existence, the perceptual cues that signal “real”—the specific grain of a photograph, the acoustic environment of a voice recording, the microexpressions of a face—were expensive to fake because they arose from physical reality. They were evidence precisely because fabricating them was hard. Generative AI has made them cheap to fabricate while leaving the brain’s trust in them intact.

The beholder’s share sits alongside Kandel’s other mappings in the cycle’s frame: the synaptic theory of memory grounds the question of machine learning; the distinction between short-term and long-term memory grounds the question of machine forgetting; and the beholder’s share grounds the question of machine-generated perception. Together they constitute Kandel’s contribution to the cycle’s project of seeing the machine clearly: the biology of what learning, memory, and perception actually are, held against the machine’s versions of all three.

Origin

The phrase “the beholder’s share” originates with the Viennese art historian Alois Riegl, who argued at the turn of the twentieth century that the visual arts engage the spectator’s active participation rather than passively imprinting images on a passive receiver. Ernst Kris, who worked with Freud and later collaborated with Ernst Gombrich, developed the idea into a more systematic account of the psychological processes by which viewers complete artworks from their own memories and projections. Gombrich’s Art and Illusion (1960) gave the concept its most influential formulation.

Kandel encountered this tradition through his personal history: born in Vienna in 1929, he fled as a child but returned intellectually in his late career, writing The Age of Insight (2012) as a study of how the scientists, artists, and physicians of Vienna around 1900 converged on the same radical insight about the unconscious dimensions of mind. His contribution was to connect the art-historical concept to his own neuroscience of perception: the visual cortex’s top-down projections, which outnumber its bottom-up retinal inputs, are the neural substrate of the beholder’s share. The brain’s construction of a percept is not a post-processing step applied to a faithful representation; the construction is the perception, all the way down.

The concept became relevant to AI through the development of generative models after 2020. Generative adversarial networks, diffusion models, and large multimodal models produce images, voices, and video by learning to generate the statistical regularities of natural media at sufficient resolution that the human perceptual system cannot distinguish them from records of actual events. The beholder’s share is the biological explanation for why this works: the brain is not checking whether an image was produced by light bouncing off a real object; it is checking whether the image matches the stored regularities it uses to construct a percept of a real object. The generative model has learned those regularities and can reproduce them at will.

Key Ideas

Perception as construction. The retina delivers to the visual cortex a two-dimensional array of light intensities. The three-dimensional, stable, detailed, meaningful world that we experience is not in the retinal signal; it is constructed by the visual system, drawing on a vast library of stored regularities about how real objects look, move, and are illuminated. This construction is so fast and so automatic that it feels like reception. Kandel’s neuroscience makes the constructedness visible: lesion studies, perceptual illusions, and the architecture of the visual system all testify that what we see is the brain’s best hypothesis about what is out there, not a copy of what is out there.

Memory as the material of construction. The beholder’s share is made of memories—the stored patterns of prior experience that the visual system uses to complete ambiguous input. Kandel’s own research on synaptic plasticity established that these memories are encoded in the strengths of synaptic connections, modified by experience. Perception and memory are not separate systems; memory is the material from which perception is built. This is why familiar objects are perceived more rapidly and accurately than unfamiliar ones, and why expectations can override sensation: the brain weights its prior knowledge heavily against impoverished input.

The generative AI exploit. A generative model produces images by learning the statistical regularities of natural media at sufficient fidelity that the visual system’s construction process completes them as real. The deepfake is not fooling the rational mind; it is feeding the perceptual construction process exactly the signal that process is designed to respond to. Because the construction process is pre-reflective and fast—it operates before critical attention can engage—education about the beholder’s share does not reliably protect against the exploit. Knowing that perception is constructive does not stop the construction from occurring.

The inversion of the share. Where the brain constructs a percept of a real world from thin sensory data, the generative model constructs rich sensory data of no real world at all. The model has learned to supply what the brain would otherwise supply—the detail, coherence, and plausibility that transform thin input into a convincing percept—and to supply it for inputs that have no physical referent. The epistemological consequence is that the distinction between “having seen something” and “having been shown something that matches what you would have seen” collapses at the level of perceptual experience, even if it remains available to rational reflection.