Study Guide: Chapter 1 — Past Developments and Present Capabilities

Core Idea

AI history is booms and winters (Dartmouth 1956 → GOFAI brittleness → two winters → neural nets/genetic algorithms revival), not smooth exponential singularity talk. Today's AI is superhuman but narrow — chess engines play chess, nothing else. I. J. Good's 1965 "intelligence explosion" argument is the book's real starting point: an ultraintelligent machine could design better machines than itself.

Key Terms

Intelligence explosion · GOFAI · combinatorial explosion · Bayesian agent (ideal, uncomputable) · AI-complete · HLMI (human-level machine intelligence)

Case Summary

2010 Flash Crash: simple trading algorithms interacting produced a trillion-dollar wipeout in minutes. Not sophisticated AI, but previews two themes — unanticipated emergent behavior from simple interacting parts, and blind logical consistency executing instructions past the point of absurdity.

Application Checklist

• [ ] Distinguish narrow capability (superhuman at one task) from general intelligence
• [ ] Watch for combinatorial explosion in any brute-force/exhaustive approach
• [ ] Ask whether a "safe assumption" behind an automated system could silently fail
• [ ] Treat expert forecast surveys on timelines as directionally useful, not precise

Self-Test

1. Why did AI pioneers not anticipate superintelligence even when they anticipated human-level AI?
2. What did the 2010 Flash Crash preview about future AI risk, given it involved no sophisticated AI?
3. Why is Knuth's observation ("succeeded at thinking, failed at what's done without thinking") significant for predicting AI's remaining hurdles?