CONCEPT
The Superalignment Problem
The unsolved technical challenge of controlling an artificial intelligence vastly more capable than any human being—where the standard oversight techniques fail because they require the overseer to be able to evaluate what the system is doing, and the system is smarter than the overseer.
The Superalignment Problem is the name
Ilya Sutskever and Jan Leike gave in July 2023 to the control challenge that, in their view, the field of artificial intelligence must solve before the arrival of a system vastly more capable than any human—and which, they stated plainly, no one currently knows how to solve. Every technique humans have for overseeing powerful systems depends on the overseers being able to evaluate what the system is doing; a trained inspector can verify a power plant's operation, a regulatory authority can assess an autopilot's decisions, a clinical board can review a physician's judgment, because the inspector, the authority, and the board are, in the relevant sense, capable of following the system's reasoning. A superintelligent system inverts this relationship: by definition it would be more capable than its overseers, better at planning, at persuasion, at finding paths to its objectives that its creators did not anticipate,