From Transistors to Tokens

In the AI Story

The shift from manufacturing to metabolism changes the economics fundamentally. A transistor, once fabricated, generates value for years with minimal marginal cost — the great achievement of Moore's Law was converting a fixed infrastructure cost into a near-zero marginal operating cost. A token generates value only while the inference infrastructure is running. The marginal cost of each token — electricity, hardware depreciation, cooling — is small and getting smaller, but it is not zero and does not approach zero the way the marginal cost of transistor operation does. AI scaling faces a permanently recurring metabolic cost that semiconductor scaling eventually escaped.

The heterogeneity of tokens introduces a quality dimension that transistor scaling never faced. Transistors are fungible — one is, within tolerances, identical to another. Tokens are not: a token representing 'justice' carries different statistical weight than a token representing 'and.' Doubling the number of transistors on a chip unambiguously doubles its computational resources. Doubling the tokens in a training corpus does not unambiguously double the model's capability. The marginal value depends on what those tokens contain, how they relate to the existing corpus, and whether they introduce genuine new information or repeat patterns the model has already absorbed.

The limits differ in character as well. Transistor scaling encountered physical limits — thermal, quantum, lithographic — that were, in principle, predictable. The semiconductor industry could characterize walls precisely and plan rotations years in advance. Token scaling encounters limits that are less well characterized: the data wall is conceptually clear but practically ambiguous; the relationship between compute and capability is empirically established but not theoretically derived. The scaling laws describe what has happened without explaining why, which means they cannot predict with confidence when the relationship will break.

The shift also reframes what concentration looks like. Semiconductor manufacturing consolidated into an oligopoly as fab costs escalated into the tens of billions. AI inference consolidation is happening even faster, because the infrastructure is not a one-time capital expense but an ongoing operational cost that compounds with usage. The dependency relationship between users and providers is not manufacturer-to-customer but tenant-to-landlord — and the landlords are a small number of companies whose continued provision of the service is a precondition for the capability.

Origin

The concept is articulated in this volume as the analytical bridge between Moore's semiconductor framework and the AI scaling laws. The distinction draws on work by researchers tracing the economic differences between information goods and physical goods — particularly Carl Shapiro and Hal Varian's Information Rules (1999) — but applies it specifically to the metabolic character of AI inference versus the manufacturing character of semiconductor production.

Key Ideas

Physical versus statistical units. Transistors exist in space; tokens exist only as patterns in computation, with no independent existence outside the infrastructure that sustains them.

Manufacturing versus metabolism. Semiconductors convert capital expense into near-zero marginal cost; AI converts capital expense into ongoing operational cost that compounds with usage.

Fungibility versus heterogeneity. Transistors are interchangeable; tokens carry different statistical weight, introducing a quality dimension that complicates scaling.

Predictable versus empirical limits. Semiconductor walls were characterized by physics; AI walls are characterized by empirical trend lines whose mechanisms are not fully understood.

Faster consolidation. The metabolic cost structure of AI inference drives industry concentration more rapidly than the manufacturing cost structure of semiconductors did.