The Discipline of Not Knowing

Origin

The concept is articulated most fully in the [Marie Curie] on AI volume of the Orange Pill Cycle, where it is identified as the bridge between Curie’s scientific method and the deepest questions AI raises about consciousness and understanding. The framing draws on the distinction between naming a phenomenon and explaining it—a distinction Curie was meticulous about. She called her discovery “radioactivity” as a descriptive label for a measured phenomenon, not as an explanatory claim about the mechanism producing it. The mechanism, at the time, was genuinely unknown: atomic structure was poorly understood, the electron had only recently been identified, and the idea that atoms could spontaneously transform and release energy ran against the foundational assumption that matter was stable. Curie’s capacity to work productively within this ignorance—to accumulate measurements that eventually compelled a new theoretical framework without claiming the framework before it was earned—is the scientific virtue the concept names.

The AI application extends naturally from the parallel structure of the problem. The capabilities of frontier models are, in a precise sense, discovered rather than designed: researchers train a system and then probe it to learn what it can do, the way Curie boiled down ore to find what was hiding inside. The force is real, the mechanism is partly opaque, the emergent capabilities accumulate before they are understood. The temptation to resolve this mystery with confident language—“the model understands,” “the system is just curve-fitting”—is the failure the discipline guards against.

Key Ideas

Naming is not explaining. To call a system’s output “understanding” or “reasoning” is to perform the collapse the discipline forbids: it treats a behavioural observation as evidence of an inner mechanism. When a model produces text that would require understanding if a human produced it, we say the model understands—and in saying so we have decided a question that is in fact wide open. Curie’s equivalent error would have been to name radioactivity and then claim she understood why atoms emitted energy. She did not make that error. She named the phenomenon and held the mechanism as unknown for as long as the evidence required.

The discomfort of sustained uncertainty. Most people and most institutions cannot tolerate sustained uncertainty about questions that matter; they resolve the discomfort by deciding, and once decided they stop inquiring. The AI field is full of premature decisions—confident dismissals and confident alarms—each a flight from the discomfort of not knowing. Curie’s example models the possibility of bearing it, of staying with a hard question long enough to let the evidence eventually speak. This is psychologically costly and institutionally difficult. It requires resisting the pressure to produce verdicts that funders, regulators, and media demand on timelines that evidence cannot accommodate.

Safety implications. The discipline of not knowing also applies to safety assessment. The early workers with radioactivity were harmed partly because the danger was invisible and partly because the readiness of many to assert safety on the basis of absent symptoms—a form of premature resolution in the optimistic direction—prevented appropriate caution. Curie’s posture was to treat the unknown as unknown, to handle the material with caution proportioned to ignorance rather than to the absence of felt harm. The AI equivalent: the absence of visible catastrophic harm is not evidence of safety, and the assessment of long-run consequences to cognition, information ecosystems, and the distribution of power requires the kind of sustained, instrumented patience that acute institutions systematically fail to supply.