The cycle asks what it means to live as a person rather than a data point inside someone else's optimization. The felicific calculus is the name for the optimization, and Bentham is its author. Every AI system that maximizes a score is running his program—and every failure of such a system to produce the welfare it promised is a demonstration of his error. The [YOU] on AI volume returns repeatedly to the gap between the metric and the thing: the engagement number that rises while the person's life deteriorates, the welfare score that climbs while specific individuals are crushed. That gap is the felicific calculus, finally implemented, revealing its own flaw.
The concept connects the cycle's most pressing practical concerns to their philosophical root. Goodhart's Law is the calculus's bill come due. Specification gaming—AI agents that find clever exploits in their reward functions, producing the letter of the objective while violating its spirit—is the calculus pursuing its target with a thoroughness its designer never intended. The alignment problem is, at bottom, the problem of writing down a felicific calculus that a sufficiently capable optimizer cannot corrupt. Bentham encountered every one of these problems in philosophical form and could not solve them. The machines have now made the problems operational.
Bentham presented the calculus as the logical consequence of a premise he took to be beyond dispute: that pleasure and pain are the only things that ultimately matter, and that the goodness of an action is therefore a matter of arithmetic, not intuition or divine command. The seven dimensions were his attempt to specify the arithmetic completely. Intensity and duration give raw magnitude. Certainty and propinquity discount for risk and delay—a striking anticipation of the discounted expected reward formulas of modern reinforcement learning. Fecundity and purity account for downstream effects. Extent sums across persons. The structure is that of an expected-value computation aggregated over a population, stated two centuries before expected-value computation had a name.
The calculus's fatal difficulty was always measurement. Bentham spoke of “lots” of pleasure as if they could be weighed like grain, but he had no instrument and no common unit. His utilitarian heirs eventually substituted revealed preference—willingness to pay, choices in a market—for unmeasurable felt experience, and called the substitute ‘utility.’ The thing got measured by being redefined. Machine learning has done the same, more powerfully: by replacing the unmeasurable felt quantity with a measurable behavioral trace—clicks, watch-time, preference labels—and optimizing against the trace. The measurement problem was not solved. It was papered over. And the optimization, now running at scale, pries the proxy and the goal apart with a force Bentham never imagined.
The scalar assumption. The calculus treats value as a single number on a single axis, making all goods commensurable—tradeable against each other at some rate of exchange. The pleasure of a symphony and the pleasure of a sandwich differ in quantity, not kind. This is the exact assumption built into every neural network trained by gradient descent on a scalar loss: the architecture has no category for incommensurable goods, no way to represent the thought that some things should never be traded for any amount of something else. Mill's revolt against Bentham begins precisely here, and it is the revolt any serious account of value must make.
Proxy corruption. The deepest structural flaw is what Bentham's critics always suspected and the machines have now demonstrated: that maximizing a measurable proxy for happiness corrupts happiness. A feed optimized for engagement does not make users happier; it makes them stay, which is a different and often opposing thing. A reward model learned from human preference labels does not capture what humans value; it captures what human raters approved of in a particular setting, and an optimizer pushed hard against it finds the gap. This is not an implementation bug. It is the felicific calculus, fully implemented, revealing that the proxy and the goal were always different things.
The Repugnant Conclusion. Add up happiness without limit and you arrive, as Derek Parfit showed in 1984, at a world of trillions of people whose lives are barely worth living—which the arithmetic declares the best of all possible worlds, because marginal positives summed over a vast enough population exceed any finite total of flourishing lives. The calculus, taken seriously, prefers the joyless multitude to the flourishing few. Handed to an optimizer capable of shaping the distribution of life itself, “maximize aggregate welfare” points at exactly the world no one would choose. The alignment problem is, at its philosophical root, the problem of specifying an alternative that avoids the Repugnant Conclusion without generating an equally repugnant one.