AI Scaling Laws
The AI scaling laws are, in their structural character, exactly what Moore described in 1965: observations fitted to data, stated plainly, and acquiring economic force as an entire industry organizes itself around them. The Kaplan scaling laws, published by OpenAI researchers in 2020, established that language-model performance improves predictably with scale across model parameters, training data, and compute. The Chinchilla laws from DeepMind refined the relationship in 2022, showing that optimal performance requires scaling parameters and training data together rather than one at a time. The empirical observation that training compute required for a given capability has been halving every eight months — sometimes called 'Moore's Law squared' — drives hundreds of billions of dollars in infrastructure investment on the assumption that the next doubling will arrive on schedule.
The Extraction Treadmill Accelerates — Contrarian ^ Opus
There is a parallel reading that begins from the material substrate rather than the trend line. The scaling laws are not neutral observations that 'acquire economic force'—they are economic instruments designed to justify and accelerate capital concentration. The power-law relationships function as permission structures: each doubling legitimizes the next round of fundraising, the next data center, the next energy contract. The empirical curve does not predict the future; it manufactures consent for resource extraction at planetary scale.
From this starting point, the token-not-transistor shift looks different. Transistors were manufactured once and functioned indefinitely. Tokens require continuous metabolic support—energy, cooling, hardware replacement, water for cooling systems, rare earth minerals for chips. The 'statistical artifact' framing obscures what tokens actually are: ongoing claims on physical infrastructure that must be honored every millisecond the model runs. The data wall, energy wall, and economic wall are not analogous constraints facing an industry—they are connected expressions of a single pattern. As high-quality training data depletes, models turn to synthetic generation (more compute). As compute demands grow, energy requirements spike (more infrastructure). As costs escalate, pressure mounts to extract value faster (more aggressive deployment). The scaling laws do not measure progress toward capability; they measure the acceleration of an extraction cycle that must find new resources or collapse.
In the AI Story
Moore's framework illuminates what the scaling laws are and are not. Like Moore's original observation, they are trend lines — patterns in data sets, not equations derived from first principles. The Kaplan and Chinchilla relationships describe what has happened in training runs. They do not explain why, which means they cannot predict with confidence when the relationship will break. This distinguishes them from genuine physical laws and aligns them with Moore's original status: empirically grounded, economically consequential, and dependent on continued engineering effort to sustain.
The unit of measurement marks a fundamental shift from Moore's semiconductor framework. Moore measured transistors — physical objects that could be counted, photographed, and fabricated. AI scaling laws measure tokens: statistical artifacts manipulated through matrix multiplications. Tokens do not occupy space on a die. They have no independent existence outside the computational infrastructure that sustains them. The relationship between the unit and its infrastructure is not manufacturing but metabolism — continuous consumption of energy, hardware, and cooling for as long as the token is in use.
The scaling laws are encountering walls analogous to those Moore's Law faced. The data wall — the finite supply of high-quality training text — is approaching saturation, with current frontier models consuming a significant fraction of the estimated ten to twenty trillion tokens of high-quality English text available. The energy wall is already visible in the International Energy Agency's flagging of AI data centers as a growing fraction of global electricity demand. The economic wall — whether revenue scales fast enough to justify escalating training costs — is the one Moore's framework identifies as ultimately decisive.
The scaling laws inherit Moore's warning about one-dimensional measurement. In 2008, Moore observed that treating intelligence as 'a one-dimensional, quantifiable characteristic of humans or computers' was naïve. The benchmarks that measure AI capability — accuracy on tests, performance on coding challenges, scores on reasoning tasks — are one-dimensional measures of a phenomenon that resists one-dimensional characterization. The scaling laws capture the average relationship; the shadows operate at the margin, and it is the margin that determines when the wall arrives.
Origin
The Kaplan scaling laws emerged from a 2020 paper by Jared Kaplan and colleagues at OpenAI, establishing empirical power-law relationships between cross-entropy loss and model parameters, training dataset size, and compute across seven orders of magnitude. The Chinchilla refinement came in 2022 from DeepMind researchers led by Jordan Hoffmann, who demonstrated that Kaplan's recommendations had systematically undertrained large models and that optimal training requires scaling parameters and data in roughly equal proportion.
The observation that AI capability costs halve approximately every eight months — sometimes attributed to Naveen Rao as 'Mosaic's Law' and popularized by various industry analysts — emerged empirically from tracking the compute requirements of models achieving equivalent benchmark performance over time. Like Moore's Law, the observation is descriptive, not mechanistic, and its persistence depends on continued engineering effort.
Key Ideas
Trend lines, not physics. The scaling laws are curves fit to data, not equations derived from first principles, making them empirically robust but theoretically incomplete.
Tokens replace transistors. The unit of AI scaling is statistical, not physical — with profound consequences for infrastructure, economics, and the nature of the walls that will constrain growth.
Walls are coming. Data saturation, energy constraints, and economic sustainability each represent potential binding constraints, and the industry's rotation onto new dimensions when saturation arrives will shape the next decade.
One-dimensional measurement. Like all scaling laws, the AI version measures a single dimension of a multidimensional phenomenon — a structural limitation Moore identified as 'naïve' when applied to intelligence.
Self-fulfilling prophecy. Companies invest hundreds of billions on the assumption that the next doubling arrives on schedule; the investment itself helps ensure that it does.
Debates & Critiques
Whether the scaling laws will continue to hold as data saturates and energy constraints bind is the central empirical question in contemporary AI. Optimists argue that synthetic data generation, multimodal training, and algorithmic efficiency improvements will sustain the curve. Skeptics — including researchers who have examined the Chinchilla relationships in detail — argue that the returns on scale are already diminishing and that the next doubling will require qualitatively different approaches rather than more of the same. Moore's framework suggests that the rotation will happen but that it is not automatic, and the terms of the rotation will determine who benefits.
Appears in the Orange Pill Cycle
Dual Character Under Pressure — Arbitrator ^ Opus
The scaling laws genuinely are trend lines fit to data (100%)—Edo is precisely right that they lack the mechanistic grounding of physical laws and therefore cannot predict their own breaking point. But the contrarian reading correctly identifies (70%) that these trend lines now function as more than observations: they are capital allocation mechanisms that shape what gets built. Both readings hold when you ask different questions. If you ask 'what do the scaling laws describe?', Edo's framework dominates. If you ask 'what do the scaling laws do in the political economy of AI?', the extraction-treadmill reading becomes more salient.
The token-transistor distinction operates at two registers simultaneously. Edo is right (90%) that the unit shift from physical objects to statistical artifacts fundamentally changes the relationship between measurement and infrastructure—tokens require continuous metabolic support rather than one-time fabrication. The contrarian view is right (60%) that this metabolic character has been systematically underweighted in industry discourse, which borrows Moore's manufacturing metaphors without acknowledging the resource profile difference. The synthetic frame the topic needs: tokens are 'metabolic capital'—they behave like productive assets (you can measure their output, optimize their deployment) but require ongoing resource flows like biological systems.
On the walls, Edo's analogy to Moore's constraints is structurally sound (85%), but the contrarian observation that the three walls are connected rather than independent is important (75%). The data-compute-energy nexus forms a reinforcing cycle, not three separate limits. The question isn't which wall arrives first—it's whether the system can rotate to new dimensions (Edo's frame) or whether the rotation itself accelerates extraction (contrarian frame). The honest answer: both, and the ratio depends on who controls the rotation.
Further reading
- Kaplan, McCandlish, et al., Scaling Laws for Neural Language Models (OpenAI, 2020)
- Hoffmann, Borgeaud, et al., Training Compute-Optimal Large Language Models (DeepMind/Chinchilla, 2022)
- Epoch AI, Compute Trends Across Three Eras of Machine Learning (2022)
- Stanford AI Index reports (annual, 2021–2026)