CONCEPT
Intent Alignment
Paul Christiano’s deliberately narrow definition of what it means for an AI system to be aligned: the system is genuinely trying to do what its operators want—a property of intent, not competence, and the load-bearing distinction between a system that is fundamentally on our side and one whose capabilities are bent against us.
The word alignment had long been used loosely in the AI safety community—sometimes meaning “safe,” sometimes meaning “beneficial,” sometimes meaning “doing what we want,” with the senses blurring into a confused tangle that made productive disagreement nearly impossible.
Paul Christiano cut through with a precise and deliberately narrow formulation: an AI system is aligned if it is
trying to do what its operators want it to do. The emphasis falls entirely on the trying, on the system’s intent, rather than on whether it succeeds or whether what we want is wise. A system can be intent-aligned and still incompetent—failing through lack of ability while genuinely trying to help—and a system can be intent-aligned even if it assists with something foolish, because intent alignment concerns only whether the system is trying to do what we want, not whether what we want is good. By