Incommensurable Values

The assumption that values can be rendered commensurable—converted into a common currency that allows them to be compared, weighed, and traded off against each other—is not a technical detail of ethical theory. It is the foundational move that makes utilitarian calculation possible, that allows preference-satisfaction accounts of well-being to claim scientific respectability, and that underlies most AI alignment frameworks, including Stuart Russell's proposal to align AI systems with human preferences. Williams and David Wiggins argued that this assumption is false and its falsity is not a technical inconvenience but a fundamental misdescription of moral life. The goods at stake in genuine moral dilemmas are often different in kind, not merely in quantity—they belong to categories of value that cannot be converted into a common currency without ceasing to be the values they are. The value of democratized capability that AI provides cannot be traded against the value of depth