Intelligence Explosion¶
Definition¶
I. J. Good's original 1965 formulation: an "ultraintelligent machine" capable of designing even better machines than itself would trigger an "intelligence explosion," leaving human intelligence far behind. Bostrom formalizes the dynamics as rate of change in intelligence = optimization power (design effort applied) divided by recalcitrance (the system's resistance to improvement). The crossover — the point where a system's own contribution to its improvement exceeds all external input — is the pivotal threshold: past it, capability gains directly increase the power devoted to the next gain, producing exponential or faster growth.
In the Book¶
Chapter 1 introduces Good's original passage and notes the AI pioneers' curious blind spot: having strained to imagine machines reaching human intelligence, they never followed the thought to its corollary of machines exceeding it. Chapter 4 is the technical core: recalcitrance plausibly falls right around human parity — via content overhang (a system reading at machine speed could absorb the Library of Congress in weeks), hardware overhang (cheap compute scaling once software works), and architectural threshold effects (a subsystem contributing nothing until it crosses a capability threshold, then suddenly dominating). Box 4 formalizes this with a differential equation showing a thousandfold capability increase within 18 months of crossover under plausible assumptions. Chapter 5 builds directly on this dynamic to explain how even a modest early lead (months, not years) can convert into an unbridgeable decisive strategic advantage if timed near the crossover inflection point.
Why It Matters¶
The intelligence explosion is the mechanism that makes every downstream problem in the book urgent rather than merely interesting: if takeoff is fast or moderate rather than slow, there is no extended window for trial-and-error correction, negotiated international response, or gradual societal adaptation. The entire argument for solving the control problem before superintelligence exists, rather than iteratively afterward, rests on the claim that this transition is unlikely to be slow.
Connections¶
- instantiates Feedback Loops — the crossover dynamic in Ch. 4 is precisely a reinforcing loop with shrinking delay: capability gains increase optimization power, which increases capability gains, faster each cycle.
- tempers Law of Accelerating Returns — Bostrom explicitly rejects "the singularity" and smooth exponential curve-fitting as the basis for taking machine intelligence seriously (Ch. 1); the crossover model in Ch. 4 replaces smooth extrapolation with a threshold-dependent dynamic governed by recalcitrance, which can produce abrupt jumps a smooth exponential trend would not predict.
- parallels Stocks and Flows — Box 4's model treats intelligence as a stock whose growth rate depends on an inflow (optimization power) divided by a resistance term (recalcitrance), a direct systems-dynamics structure.
- extends Decisive Strategic Advantage — Ch. 5 shows how the crossover dynamic converts a modest time-lead between competing projects into total, unrecoverable separation.