Responsible Scaling Policy

In the AI Story

The history of powerful technologies is a history of missing frameworks. Nuclear energy arrived before its regulatory infrastructure. The automobile arrived before traffic laws and seat belts. The internet arrived before privacy law. In each case, the technology arrived first, the consequences arrived second, and governance frameworks arrived third — too late to prevent the harms earlier governance could have mitigated. Amodei studied this pattern and concluded that the AI industry was positioned to repeat it unless the industry itself took the initiative to build frameworks in advance of the harms they were designed to prevent. As Edo Segal observes, governance arrives eighteen months after the tools it was meant to govern — and for AI, the lag would be measured in years.

The biosafety analogy is deliberate and illuminating. In biological research, the level of containment required for working with a pathogen is determined by its characteristics: transmissibility, virulence, available treatments, consequences of accidental release. The containment level is proportional to the risk, and the determination of the risk precedes the work rather than following the consequences. This prospective approach to risk management was precisely what Amodei wanted to establish for AI development — a framework in which safety measures required for working with a model were determined by the model's capabilities, assessed before deployment, not by the consequences of deployment, assessed after the harm had occurred.

The practical implementation required organizational discipline most technology companies would find unfamiliar. Before deploying a new model, Anthropic's safety team conducts evaluations that test capabilities against thresholds defined in the framework. If evaluations reveal the system has crossed into a new capability tier, additional safety measures are required before deployment proceeds. These might include additional red-teaming, monitoring infrastructure, usage restrictions, or — in extreme cases — a decision not to deploy until safety measures are adequate. The decision not to deploy was the hardest because it cost the most: a system ready from a capability perspective but not from a safety perspective was a system competitors would deploy first.

The framework also required investment in evaluation methods that did not yet exist. Many risks were theoretical rather than observed. No AI system had yet been used to develop a biological weapon or conduct an autonomous cyberattack. The evaluations had to test for capabilities that might enable these outcomes, requiring a combination of technical expertise, adversarial creativity, and willingness to imagine scenarios the designers hoped would never materialize. The red-teaming process was itself a form of safety research, producing knowledge about potential failure modes that informed both deployment decisions and the design of subsequent systems.

Origin

The RSP was published by Anthropic in September 2023 as version 1.0 and has been iteratively revised since. It represented the first public commitment of its kind by a frontier AI lab and served as a template that other organizations — OpenAI, Google DeepMind — subsequently adapted for their own frameworks.

Amodei designed the framework to be both binding on Anthropic and replicable across the industry, arguing that voluntary frameworks are complements to, not substitutes for, government regulation. In November 2025, on 60 Minutes, he publicly called for mandatory regulation — a CEO advocating the constraint of his own company because he believed the constraint was necessary.

Key Ideas

Prospective not reactive. The framework anticipates risks associated with future capability levels and specifies required safety measures before the capability is achieved, rather than waiting for incidents to trigger measures.

Capability and safety co-evaluated. A system's readiness for deployment is determined by the relationship between its capability and its safety infrastructure — not by either alone.

Binding, not advisory. Thresholds are commitments the organization makes that constrain behavior in ways that may cost revenue. A framework that can be suspended under commercial pressure is not a framework.

Published risk categories. The framework specifies evaluations for biological weapons assistance, cyberattack capability, autonomous behavior, and manipulation — making the governance structure testable and subject to external scrutiny.

Voluntary complements to regulation. Amodei argues that industry frameworks demonstrate what responsible self-governance looks like but cannot substitute for government regulation that applies equally to all participants.

Debates & Critiques

Critics argue that voluntary frameworks are inherently unstable — that any company can suspend its own policy under sufficient competitive pressure, and that the thresholds themselves are set by the companies that would be constrained by them. Defenders point to the RSP as the first serious attempt at prospective technology governance and argue that industry leadership can create the political space for mandatory standards. The deeper debate concerns whether capability thresholds can be meaningfully defined in advance for systems whose behaviors emerge from training rather than being designed.