The Stochastic Parrot

A stochastic parrot is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning. The phrase was coined by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell in their 2021 paper “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”—a paper whose title became one of the most consequential acts of naming in the history of artificial intelligence. The word stochastic means random in a patterned, probability-governed way; the word parrot points at mimicry without comprehension: a bird that reproduces human speech with startling fidelity while grasping none of what it says. Bender’s claim is that a large language model, however more sophisticated than a bird, sits on the same side of a crucial divide. The fluency is a property of the statistics, not a sign of a mind behind them. The text