Purpose vs. Optimization

“We don’t optimize,” Oliver Selfridge said near the end of his life, “we improve. To me that should be part of the essence of AI.” The sentence sounds gentle. It is a quiet bomb planted under the entire contemporary paradigm. To optimize is to drive toward the best value of a target that is given and held still—a static universe with a known summit, where the only task is to climb. Selfridge thought that picture was false to how real intelligence works. Real agents improve, provisionally, toward purposes that themselves keep changing as circumstances and understanding shift. The mathematical treatments he was skeptical of are looking for formalistic presentations of processes that can then be optimized, but the world is not a fixed formalism with a stable peak. Improvement is doing a little better than last time, toward an end you may revise tomorrow; that is what people do, and that, he thought, is what a real intelligence