Biosafety Levels

In the AI Story

Amodei and Anthropic's team chose biosafety as the explicit analogy for AI safety thresholds because the framework embodies three principles they wanted to establish for AI development. First, risk management is prospective — the containment level required for working with a pathogen is determined before the work begins, based on characteristics known in advance. Second, the framework is graduated — different levels of risk require different levels of containment, rather than a single binary safe/unsafe determination. Third, the framework is binding — laboratories working with dangerous pathogens must implement the specified containment measures as a condition of conducting the research.

The biosafety framework evolved from specific disasters and near-disasters in mid-twentieth-century biomedical research. The 1977 smallpox escape from a Birmingham laboratory, the 1979 Sverdlovsk anthrax leak, and earlier incidents demonstrated that biological research could produce catastrophic outcomes if conducted without adequate containment. The framework that emerged from these events was not a theoretical construction but a pragmatic response to demonstrated failures — a feature Amodei valued as he considered how to apply similar discipline to AI.

The analogy has limits that Amodei has acknowledged. Pathogens have characteristics that can be measured with relative precision; AI capabilities are harder to evaluate. Pathogens do not strategically manipulate their containment; sufficiently capable AI systems might. Pathogens do not provide positive value that must be balanced against containment costs; AI systems do. These disanalogies mean the biosafety framework cannot be applied mechanically to AI, but its structural features — prospective assessment, graduated response, binding requirements — remain valuable even where the specifics differ.

The institutional legitimacy of biosafety provides another dimension of the analogy's value. Biomedical research is conducted within a framework of regulation, professional norms, and institutional review that constrains what researchers can do while permitting valuable work to proceed. The framework is not perfect — breaches occur, enforcement is uneven — but it represents a genuine attempt to manage catastrophic risks prospectively. Amodei's argument is that AI development needs an analogous framework, developed before the catastrophic incidents rather than after.

Origin

The biosafety framework developed through multiple institutional efforts, including the Centers for Disease Control and Prevention (CDC), the National Institutes of Health (NIH), and the World Health Organization (WHO). The current four-tier classification system was formalized in the 1984 edition of the CDC-NIH manual Biosafety in Microbiological and Biomedical Laboratories, which has been periodically updated since.

The framework's influence extends beyond biology into other domains of high-risk research. Nuclear facilities, radiation laboratories, and other hazardous research environments have adopted analogous tier-based classification schemes. The AI Safety Levels introduced by Anthropic in 2023 represent the extension of this approach to a new domain.

Key Ideas

Four tiers of containment. BSL-1 through BSL-4 match containment measures to the risks posed by specific pathogens, from standard laboratory practice to positive-pressure suits and airlocked facilities.

Prospective not reactive. The containment level is determined before research begins, based on known characteristics of the pathogen.

Institutional legitimacy. The framework exists within a broader regulatory and professional structure that gives it practical force.

Disanalogies with AI. Pathogens do not strategically manipulate containment; AI might. Pathogens do not provide positive value; AI does.

Structural lessons nonetheless. The prospective assessment, graduated response, and binding requirements of biosafety remain valuable models for AI governance even where the specifics differ.

Debates & Critiques

The debate about the biosafety analogy for AI concerns both its applicability and its sufficiency. Some argue the analogy is too strong — that AI capabilities cannot be evaluated with the precision that pathogen characteristics allow. Others argue it is too weak — that AI's potential to be used against human interests makes it more like a dual-use weapon than a pathogen. Defenders argue that the analogy is productive precisely because it forces explicit consideration of these differences and focuses the conversation on structural features of prospective risk management.