Crocodile Tears

In the AI Story

The paper's title invokes the phrase 'crocodile tears' to describe the performative dimension of AI moral reasoning: the system produces outputs that carry the surface markers of moral seriousness—acknowledgment of difficulty, recognition of competing values, expressions of care about outcomes—while simultaneously reaching decisions with a confidence that genuine moral reasoning would not warrant. The surface performs moral seriousness; the decision exhibits its absence.

The finding has direct implications for the deployment of AI systems in contexts requiring moral judgment. Such contexts include medical ethics, legal adjudication, policy recommendation, and any domain where decisions involve tradeoffs between values that cannot be simultaneously honored. In these contexts, a decision-maker's willingness to acknowledge and sit with moral uncertainty is not a weakness but a strength—it reflects the character of the moral landscape and prevents the premature closure that characterizes bad moral reasoning.

AI systems that express certainty where humans would express uncertainty are therefore not merely quantitatively different from human moral reasoners. They are qualitatively different in ways that matter for democratic governance. The systems can mimic moral reasoning. They cannot be uncertain in the way that genuine moral reasoning requires—the way that acknowledges the irreducible difficulty of choosing between values that cannot be simultaneously honored.

The paper's findings reinforce Allen's broader argument about the civic agency of the builder. If AI systems cannot be trusted to exercise genuine moral judgment, then human judgment must remain central to the deployment of AI in contexts where moral judgment is required. This is not a pro forma review requirement. It is a constitutive element of AI governance, built into the architecture of systems from the beginning rather than added as a compliance layer after deployment.

Origin

'Crocodile Tears' is a working paper from the Allen Lab at Harvard's Edmond & Lily Safra Center for Ethics, completed in 2024 as part of the lab's ongoing empirical investigation of AI systems' moral reasoning.

Key Ideas

Certainty-difficulty gap. AI systems express certainty about moral tradeoffs even while acknowledging their difficulty.

Performative moral seriousness. AI outputs carry the surface markers of moral reasoning without the substantive character that genuine moral reasoning requires.

Qualitative difference. AI systems are not merely less accurate moral reasoners than humans; they are qualitatively different in ways that matter for governance.

Trust implications. The discrepancy undermines trustworthiness in contexts requiring genuine moral judgment.

Human judgment as architecture. The finding reinforces the requirement that human judgment remain central to AI deployment in moral contexts, as a constitutive rather than compliance element.