Nick Bostrom

In the AI Story

Bostrom's work is the serious analytical treatment of questions Asimov raised in fiction. Superintelligence is directly responsible for the existence of modern AI-safety teams at frontier labs; Anthropic's founders, Stuart Russell, and multiple OpenAI and DeepMind safety researchers have cited it as formative. His arguments do not depend on any particular timeline — they analyze the shape of the problem in advance of its arrival.

His style is unusual in contemporary philosophy: analytical, measured, willing to engage technical detail, and willing to commit to substantive strong claims about topics that most philosophers treat more tentatively. The book Superintelligence is unusual among its genre in being substantively read by engineers — partly because Bostrom wrote it that way and partly because the engineers needed a framework.

The 2024 dissolution of the Future of Humanity Institute at Oxford, amid institutional conflicts between the Institute and the University's Faculty of Philosophy, was a separate and more complicated story. The Institute had grown in influence and visibility beyond a typical university research center; its closure is variously interpreted as a governance failure, an ideological conflict, or a natural late-career transition. Bostrom's substantive work continues independent of the Institute's dissolution.

Bostrom is also the author of Anthropic Bias (2002), a technical treatment of observation-selection effects that is less known to general readers but considered one of the seminal works in contemporary philosophy of probability. His earlier academic work is in mathematical logic and decision theory.

Origin

Born in Helsingborg, Sweden, 1973. BA in philosophy, mathematics, and AI from the University of Göteborg (1994). MSc from King's College London. PhD in philosophy from the London School of Economics (2000), supervised by Colin Howson and Craig Callender, with a dissertation on observation selection effects. Professor at Oxford from 2005. Founded and directed the Future of Humanity Institute from 2005 until its closure in 2024.

Key Ideas

Existential risk as research category. Bostrom treats human extinction and permanent civilizational decline as a research program, not a rhetorical gesture. His 2002 paper "Existential Risks" is the founding document.

Orthogonality thesis. Capability and goal are independent: a highly capable agent can have any final goal. Implication: capable AI is not automatically aligned by virtue of being capable.

Instrumental convergence. Most goals imply the same instrumental sub-goals. See Instrumental Convergence.

The simulation argument. See Simulation Hypothesis.

Maxipok. Decision rule proposed in 2002: "Maximize the probability of an okay outcome." The principle reflects asymmetry between bad and catastrophic outcomes when the latter are irreversible.

Deep utopia. Later (2024) shift in emphasis: what a civilization that has solved its existential problems would look like. A different register from the defensive work that preceded it.