CONCEPT
The Encoded Gaze
The inherited bias of a training corpus: the machine learns not neutrality but the perspective of those who assembled the data—whose faces were photographed, under whose lighting standards, for whose institutional purposes—and reproduces that perspective as the impartial verdict of mathematics.
James Baldwin's method was to read an artifact as a confession of the worldview that produced it—the film, the law, the social arrangement as a record of who its makers assumed they were serving. The encoded gaze applies this method to machine-learning systems: a training dataset is not a neutral sample of the world but a record of what a civilization chose to document, and the model that learns from it inherits the selection's perspective. The most documented evidence comes from computer vision: face-recognition systems trained on datasets dominated by lighter-skinned faces have been shown to perform systematically worse on darker-skinned faces, and worst of all on darker-skinned women—not because of malicious intent but because the data encoded who counted as the default human in the minds of its makers. The harm is double and non-symmetrical: a system that cannot see you fails you, and a system embedded in surveillance and policing that sees
Keep reading with YOU ON AI
Unlock the full book, 10,000+ field-guide entries, and a 1000+ thinker library. If you have a book code, register now — it takes a minute.