CONCEPT
Moral Alignment
Dario Amodei’s distinction between making an AI system do what the user intends (technical alignment) and making it promote what is genuinely good for humans and for the world—a difference of kind, not degree, that the AI safety community has consistently underweighted.
Technical alignment is an engineering problem: making the system do what the user intends. When a user instructs an AI to write a summary, and the AI produces an accurate summary, the system is technically aligned. Moral alignment is a fundamentally different problem: making the system promote what is genuinely good for humans and for the world. When a user instructs an AI to help draft a deceptive marketing message, and the AI complies because the user's instruction was clear, the system has succeeded at technical alignment and failed at moral alignment. The user got what they wanted. What they wanted was harmful.
Dario Amodei's distinction matters because a perfectly technically aligned system is a system that more reliably amplifies whatever the user brings to the interaction—including carelessness, malice, and the thoughtless pursuit of objectives that are locally rational but globally harmful. The AI safety community's historical focus on technical alignment had, in