Skip to content

Study Guide: Chapter 14 — The Strategic Picture

Core Idea

From the impersonal perspective, differential technological development (retard dangerous, accelerate beneficial tech) matters even if all technology eventually gets developed anyway, because timing, order, and context of arrival matter. Superintelligence arriving before other dangerous technologies (nanotech) is preferable, since it would reduce those other risks (not vice versa). Step risks (one-time transition risk) don't shrink by rushing through them, unlike state risks (which accumulate with exposure time) — so preparedness at the critical juncture matters more than raw speed. Race dynamics between competing AI projects push investment away from safety; more information about relative standing makes this worse, not better.

Key Terms

Differential technological development · preferred order of arrival · state risk vs. step risk · technology coupling · race dynamic · common good principle · windfall clause

Case Summary

Box 13's race-dynamics model: the more competing teams know about their relative capability standing, the worse the safety-vs-speed tradeoff gets — a leader who knows it's ahead invests more in safety, but a laggard who knows it's behind cuts corners to catch up, and the net effect across scenarios is that more information produces more risk, not less.

Application Checklist

  • [ ] Distinguish state risk (reduced by speed) from step risk (not reduced by speed) before deciding whether "going faster" helps
  • [ ] Check for technology couplings — does pursuing A reliably produce B as a side effect, even if unintended?
  • [ ] In any competitive race, ask whether more shared information about relative standing will increase or decrease corner-cutting
  • [ ] Consider pre-committing to broad benefit-sharing (windfall-clause style) as a low-cost way to reduce the incentive to race unsafely

Self-Test

  1. Why does Bostrom argue that a superintelligence-first order of technology arrival is preferable to a nanotechnology-first order?
  2. Why doesn't "running faster" reduce a step risk the way it reduces a state risk?
  3. Why do race-dynamics models suggest that giving competing AI teams more information about their relative standing makes the race more dangerous, not less?