CONCEPT

The Boltzmann Machine

The stochastic, energy-based neural network that Sejnowski and Hinton built in 1985—learning by closing the gap between what it dreamed on its own and what the world had shown it—and the conceptual ancestor of today's generative models.

In 1985, Terrence Sejnowski, Geoffrey Hinton, and David Ackley published a paper in Cognitive Science introducing an object that borrowed its name from the nineteenth-century physicist Ludwig Boltzmann and its insight from statistical physics. A Boltzmann machine is a network of binary units—each either on or off—connected symmetrically, and the crucial novelty is that these units decide their states stochastically, flipping with a probability that depends on their inputs. The network is noisy on purpose, and this deliberate noise, borrowed from the way physical systems behave at a temperature, is the key to making such a network learn. Just as a warm physical system settles toward low-energy configurations while never freezing into the single lowest one, a Boltzmann machine wanders its landscape of possible states and can be gradually cooled—a procedure called simulated annealing—to drift toward good configurations without getting stuck. The learning algorithm is of unusual elegance: run the network in two phases, one with data clamped

Keep reading with YOU ON AI