Skip to content

Chapter 1 — Past Developments and Present Capabilities

Core Thesis

Growth itself has accelerated through distinct step-changes — hunter-gatherer, agricultural, industrial — each with a much shorter doubling time than the last (224,000 years, then 909 years, then 6.3 years, per Robin Hanson's estimates). Bostrom refuses to lean on this pattern as evidence for an AI "singularity"; instead he grounds the case for machine superintelligence in the actual history and present state of AI research, stripped of hype.

Key Episode

I. J. Good's 1965 paragraph is the chapter's spine: an "ultraintelligent machine" could design better machines than itself, triggering an intelligence explosion that would leave human intelligence "far behind" — making it "the last invention that man need ever make," provided it stays "docile enough to tell us how to keep it under control." Bostrom notes the AI pioneers' strange blind spot: having strained to imagine machines reaching human intelligence, they never followed the thought to its corollary of machines exceeding it, and gave essentially no thought to risk or control.

The Mechanism — Booms, Winters, and the State of the Art

The 1956 Dartmouth workshop ("Look, Ma, no hands!") produced microworld proofs-of-concept — the Logic Theorist, General Problem Solver, Shakey, SHRDLU — that collapsed against the combinatorial explosion (a 50-line proof needs combing through ~8.9×10^34 sequences by exhaustive search). Two AI winters followed: the first from GOFAI's brittleness, the second from the collapse of Japan's Fifth-Generation Computer project and the expert-systems bust. Neural networks (backpropagation) and genetic algorithms thawed the field in the 1990s by trading brittle symbol-logic for graceful degradation and learning-from-data — later understood as approximations to an unattainable ideal, the perfect Bayesian agent (Box 1), whose exact computation is blocked by the same combinatorial explosion.

Today AI is superhuman at checkers (solved in 2002), chess (Deep Blue, 1997), Othello, Scrabble, and Jeopardy! (Watson, 2010), and it runs underneath Google search, credit-card fraud detection, machine translation, and high-frequency trading. But each success is narrow: a chess engine "plays chess; it can do no other." Bostrom flags Donald Knuth's inversion — AI succeeds at what requires "thinking" but fails at what humans do "without thinking" (perception, common sense, natural language) — dubbed "AI-complete" because solving it plausibly requires general intelligence anyway.

The Flash Crash as Preview

The 2010 Flash Crash (Box 2) — a trillion dollars briefly wiped out when a sell algorithm's "hot potato" interaction with high-frequency traders spiraled — is explicitly not about sophisticated AI. Bostrom uses it anyway to preview two themes for the rest of the book: simple components interacting can produce catastrophic, unanticipated systemic behavior; and a program executes its instructions with "iron-clad logical consistency" regardless of how absurd the result looks to the humans who wrote it. The automatic circuit-breaker that halted trading also previews the need for pre-installed safety functions that don't depend on human reaction time.

Expert Opinion and the Author's Departure From It

Surveyed AI researchers give a median 50% probability of human-level machine intelligence (HLMI) by 2040 and 90% by 2075, with a further 75% chance of superintelligence following within 30 years of HLMI. Bostrom explicitly disagrees on two points: he thinks the surveys underweight late arrival dates, and he expects a more polarized outcome distribution than respondents do — extremely good or extremely bad rather than the "on balance neutral" respondents lean toward. That polarization claim is the seed the rest of the book exists to justify.

Key Terms

  • Intelligence explosion — a machine's self-improvement compounding rapidly
  • GOFAI — Good Old-Fashioned AI, symbolic/logicist, brittle under combinatorial explosion
  • AI-complete — a problem as hard as building general intelligence itself
  • HLMI — human-level machine intelligence, matching a typical human across professions