Reid Hoffman on AI · Ch6. The Speed Question and the Stakes Question ← Ch5 Ch7 →
Txt Low Med High
PART TWO — Speed, Stakes, and the Blitzscaling of Intelligence
Chapter 6

The Speed Question and the Stakes Question

Page 1 · The Speed Question and
Ai Alignment
Ai Alignment

The deepest argument inside Blitzscaling, and the one that matters most for AI, is the relationship between speed and stakes. In low-stakes markets, speed wins because the cost of failure is small. In high-stakes markets, speed and stakes can pull in opposite directions, and the question becomes whether the gains from speed exceed the costs of imperfect deployment. Hoffman has consistently argued that AI, despite its high stakes, is more like a low-stakes market in this sense — that the gains from getting capable systems into the world quickly exceed the costs, and that the costs are themselves reduced by being in the world rather than in the lab.

The argument against is straightforward: AI is not like a social network or a ride-sharing app, because the failure modes are not bounded.

This is not a universally held view, even among the AI optimists. The argument against is straightforward: AI is not like a social network or a ride-sharing app, because the failure modes are not bounded. A poorly tuned recommendation algorithm produces engagement-maximizing slop. A poorly aligned foundation model could, in worst-case scenarios, produce bioweapons recipes, undetectable disinformation, or autonomous systems whose behavior diverges from their specification in ways the developers cannot diagnose. The asymmetry between gains and losses changes the calculation. You cannot recall a model the way you can recall a feature.

Hoffman's response has several layers. First, he argues that the worst-case scenarios are usually less plausible than the doomers claim — that current systems are not on a trajectory to autonomous deception, and that the alignment problem, while real, is being made tractable by the same iteration process that produces capability gains. Second, he argues that the benefits of AI are not speculative but already being realized — in education, in healthcare, in scientific research — and that delay has real costs measured in lives not saved and problems not solved. Third, he argues that geopolitical dynamics make unilateral slowdowns counterproductive, because the technology will be developed regardless and the question is only who develops it.

· · ·
Page 2 · The Speed Question and

Each of these arguments has weaknesses. The first depends on a confident reading of current alignment research that not all researchers share. The second is true but harder to weigh when the harms are diffuse and the benefits concentrated. The third has a long history of being used to justify dangerous accelerations whose costs were borne by people who did not get to vote on them. Hoffman would say that his point is not that there are no costs but that the alternatives — centralized control, geopolitical retreat, indefinite pause — have worse costs and lower probabilities of being achieved.

Nassim Nicholas Taleb
"The difference between a good and a bad doctor is that the bad one does not know what he doesn't know. The same difference applies to those who claim to know the risk profile of frontier AI."
Antifragile · 2012

What is genuinely distinctive in Hoffman's position is that he is willing to keep updating. He has been clear that he would slow down on specific capabilities if specific evidence warranted it. He has been involved in efforts to red-team frontier models and to develop evaluation frameworks. He has argued that the labs should be more transparent about their safety practices, and he has criticized labs he sees as cutting corners. This is not a doctrinaire accelerationism. It is, at its best, a wager that speed plus steering beats either alone.

The wager could be wrong. Hoffman knows this. The orange pill, in his version, is the willingness to act in the face of irreducible uncertainty rather than to wait for certainty that will not arrive. He is not arguing that the future is bright. He is arguing that paralysis is the worst response to a future that is going to happen with or without you. The question is whether the future he is building toward — one in which capable AI is widely deployed, embedded in democratic institutions, and steered by distributed feedback — is the one that actually arrives, or whether the same forces that produce the speed will overwhelm the steering.

· · ·
Nassim Nicholas Taleb
Further Reading From The Orange Pill Cycle · Related Thinkers
6 voices alongside this chapter — click to meet them
Continue · Chapter 7
The Techno-Humanist Compass
← Prev 0%
Ch6 Next →