Cargo Cult Productivity

In the AI Story

The original cargo cult observation was ethnographic. Feynman adapted it into a concept he called cargo cult science in his 1974 Caltech commencement address — research with the form of science but not the substance, that follows the procedures without understanding the principles that make those procedures meaningful. Publications look like scientific papers but do not advance understanding, because the researchers missed the principles of experimental design and hypothesis testing that give methodology its power. They reproduced the form. They missed the substance.

Applied to AI-assisted work, the pattern operates at industrial scale. The triumphalists of the Orange Pill moment post metrics with athletic enthusiasm: lines of code generated, applications shipped, pull requests merged, revenue achieved. The numbers are extraordinary. The output is visible, measurable, shareable. It has all the observable features of productivity. Some of it is genuine — real work solving real problems for real people. Some of it is the reproduction of the form without the substance: output following the patterns of valuable work without containing the judgment, understanding, and deep engagement that make work valuable.

The Berkeley study provides empirical support. Ye and Ranganathan's eight-month ethnography found that AI intensifies work without necessarily improving it — workers faster, taking on more tasks, crossing role boundaries, with output increasing by every measurable metric. The study could not determine whether the additional work was better or worse than what it replaced. More output was produced. Whether that output represented genuine progress or merely increased volume remained an open question. The form of productivity was present. The substance was unmeasured.

The deeper danger is not that cargo cult productivity produces bad work. It is that it erodes the capacity to distinguish good work from bad. Feynman's original concern was that when the form of science is replicated without the substance, the community's ability to recognize genuine science diminishes — the distinction depends on understanding the principles, not just the procedures. When the form of productive output is replicated at scale, the signal-to-noise ratio changes: not because signal has weakened, but because noise has become indistinguishable from signal. This is the epistemological crisis AI creates for human knowledge work, and it compounds Sagan's 1996 warning about a civilization dependent on technologies it does not understand.

Origin

The phrase 'cargo cult' entered English-language anthropology in the 1940s through descriptions of the John Frum movement on Tanna and similar millenarian movements across Melanesia. Feynman encountered the concept and adapted it for science in his 1974 Caltech commencement address, published in 1985 in Surely You're Joking, Mr. Feynman!

The Sagan volume extends Feynman's concept to AI-assisted work, making explicit what was implicit in The Orange Pill's account of the Berkeley study and Segal's own observations about engineers whose productivity multiplier disguised the loss of the formative ten minutes of architectural insight buried in each four-hour block of plumbing work.

Key Ideas

Form without substance. The bamboo airstrip has every observable feature of an airstrip and none of the connection to the global aviation system that makes airstrips functional.

Visibility as trap. Productivity metrics measure what is measurable, which tends to be form rather than substance; this creates incentive structures that reward cargo cult output.

Signal-noise collapse. When the form of competent work is replicated at scale, the distinction between genuine competence and its simulation becomes harder to perceive, not because signal weakens but because noise matches it.

Erosion of discrimination. The most insidious harm is not bad output but the atrophy of the capacity to tell good output from bad — a capacity that depends on the understanding that cargo cult production bypasses.

Feynman's warning generalized. The original concern was scientific methodology; the expanded concern is any domain in which form and substance can come apart, which includes nearly all knowledge work.

Debates & Critiques

Defenders of AI-assisted work argue that the cargo cult framework unfairly stigmatizes outputs that function successfully, regardless of whether their producer understands them deeply. The Sagan volume's counter-argument is that functional success in the short term is compatible with systemic erosion of understanding in the long term, and that the long-term erosion is the harder problem to see and the more dangerous one to ignore.