The Generation-Evaluation Asymmetry
Boden's framework makes visible a distinction the AI discourse routinely collapses: the difference between producing candidate creative outputs and evaluating which outputs are valuable. The machine performs the first operation at extraordinary scale and speed. It can generate thousands of plausible combinations, millions of coherent sentences, billions of possible chess positions. But it cannot — in any sense Boden recognizes as substantive — evaluate which of its outputs are genuinely illuminating, which are merely plausible, and which are superficially clever fabrications that collapse under examination. Evaluation requires taste, domain depth, stakes in the world, and the capacity to recognize quality that has not been defined in advance. These remain human contributions, and they constitute the scarcity that makes human participation in AI-augmented creative work continue to matter.
The Evaluation Commodification Thesis — Contrarian ^ Opus
There is a parallel reading that begins not with the permanence of judgment scarcity but with the mechanisms already eroding it. The asymmetry Boden identifies may be structural in principle while proving temporary in practice—not because AI develops consciousness, but because evaluation itself is being re-encoded as a tractable computational problem.
Consider what 'evaluation' actually means in most professional contexts. A lawyer evaluates whether an argument will persuade a specific judge based on precedent and rhetorical patterns. A marketer evaluates whether copy will convert based on prior campaign data and audience response curves. A manager evaluates whether a strategy will work based on comparable past initiatives and organizational dynamics. In each case, what presents as irreducible judgment is substantially pattern-matching against a corpus of prior evaluations and their outcomes. This is precisely what large language models trained on evaluation datasets—code reviews, editorial decisions, strategic assessments—are learning to approximate. The 'stakes' that supposedly distinguish human evaluation may be less about genuine caring and more about having sufficient training examples of what counted as good in similar contexts. As AI systems are trained not just on outputs but on the full evaluation discourse around those outputs—the red-penned drafts, the revision histories, the accepted/rejected pairs—they are building precisely the evaluative corpora Boden claims they lack. The asymmetry persists only until evaluation itself becomes sufficiently well-documented to serve as training data.
In the AI Story
The asymmetry is not a contingent feature of current AI systems — a limitation that will be overcome with more parameters and training data. It is structural. Evaluation, in the sense Boden means, requires a subject who is surprised, who recognizes value, who cares about outcomes. Whether AI can acquire such subjectivity remains the deepest open question in the field; current systems have not demonstrated it.
The practical consequence is immediate. A builder using Claude can produce ten drafts of an essay in an hour. Each draft is coherent, each uses language fluently, each organizes arguments plausibly. The quality-determining work is not the production of the drafts but the selection among them — recognizing which arguments actually hold, which prose carries genuine weight, which connections are illuminating rather than merely clever. This work cannot be delegated to the machine because the machine does not know what to select for.
The asymmetry explains why the new scarcity in the AI age is judgment rather than execution. Execution has commoditized — anyone with a subscription can generate competent-seeming output across essentially every codified domain. Judgment — the evaluative capacity that separates the genuinely valuable from the superficially plausible — remains rare, cannot be directly purchased, and is cultivated only through the years of deep engagement with a domain that deliberate practice requires.
The asymmetry also explains the aesthetics of smoothness danger. AI output arrives polished, coherent, authoritative-seeming. Without evaluative discipline, the polish is mistaken for substance. The builder accepts what sounds good. Over time, her capacity to distinguish sound from substance atrophies. The Deleuze failure is one instance; every AI-augmented knowledge worker faces the same risk daily.
Origin
The asymmetry is implicit in Boden's taxonomy from its first formulation but received its clearest articulation in her later work engaging with contemporary AI systems. The distinction between generation and evaluation becomes more consequential as generation capacity expands and evaluation remains the human-specific contribution.
Key Ideas
Generation and evaluation are structurally different. The machine can do the first at superhuman scale; it cannot, in any substantive sense, do the second.
Evaluation requires stakes. Recognizing value demands a subject who cares about outcomes — the very capacity machines have not demonstrated.
The new scarcity is judgment. As execution commoditizes through AI tools, evaluative capacity becomes the scarce resource that makes human participation worth something.
Smoothness is the danger. AI output's polish disguises the need for evaluation; builders who do not cultivate discipline will mistake plausibility for substance.
Judgment cannot be bought. Unlike execution, which has a market price, evaluative capacity is cultivated only through years of deep domain engagement.
Debates & Critiques
The core contested question: can AI develop evaluative capacity? Enthusiasts argue that reinforcement learning from human feedback has already given models proto-evaluative capabilities. Skeptics argue that mimicking human evaluation patterns is not the same as evaluating, that genuine evaluation requires stakes machines do not have. Boden's position tracks the skeptical view while acknowledging the question as open.
Appears in the Orange Pill Cycle
Domain-Dependent Evaluation Gradients — Arbitrator ^ Opus
The right weighting depends entirely on what type of evaluation we're examining. For rule-bound evaluation—code correctness, logical validity, adherence to style guides—Boden's asymmetry is already collapsing (80% toward the contrarian view). AI systems now catch bugs, identify logical fallacies, and enforce consistency with increasing reliability. The 'judgment' here was always semi-algorithmic pattern-matching, and sufficient training data has made it tractable.
For taste-dependent evaluation in established domains—literary quality within a genre tradition, strategic soundness in known business contexts, persuasiveness for documented audiences—the picture splits (60/40 favoring the contrarian thesis). What appears as irreducible judgment often decomposes into learnable heuristics once you have enough evaluation examples. The chess position evaluator seemed to require deep understanding until we had enough annotated games. Similarly, an editor's 'eye' for compelling prose or a strategist's 'nose' for workable plans may be more trainable than Boden suggests, given sufficient documented evaluations.
But for value-recognition in genuinely novel spaces—work that transforms the criteria of evaluation itself, contributions whose significance won't be clear for years, ideas that shift what counts as good—Boden's structural asymmetry holds completely (95% toward the original entry). Here evaluation isn't pattern-matching but stakes-laden judgment about what matters, and stakes require being a subject with skin in the game. The synthesis: evaluation commodifies domain by domain as documentation accumulates, but the frontier of value-creation—where the rules of evaluation are themselves being rewritten—remains irreducibly human.
Further reading
- Margaret Boden, The Creative Mind, Chapters 1-2
- Margaret Boden, Creativity and Art, Chapter 5