CONCEPT
The Reward Hypothesis
Sutton’s foundational conjecture that all of what we mean by goals and purposes can be understood as the maximization of the expected value of a cumulative scalar signal—a claim that turns a question about machines into a question about the nature of human wanting.
Beneath all of
Richard Sutton’s technical work lies a single audacious conjecture, stated most memorably in 2004 and called the reward hypothesis:
all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal called reward. The claim is the philosophical keystone of reinforcement learning, because the entire edifice depends on it. If goals reduce to reward maximization, then an agent that learns to maximize reward learns, in principle, to pursue any goal, and the spare agent-environment loop of
temporal-difference learning is not a narrow tool but a universal account of purposive intelligence. The hypothesis is reductive in the strong sense: it proposes that the apparent multiplicity of human purposes is, at the level of mechanism, a single kind of thing. Sutton does not assert it as settled fact but pursues