Skip to content

Chapter 15 — Crunch Time

Core Thesis

Closing the book under "a dense mist of uncertainty," Bostrom asks the practical question directly: what should we actually do, given everything argued so far? His answer rejects two easy postures — paralysis and business-as-usual — for a specific triage: focus effort on problems that are urgent, robustly positive-value, and elastic to additional effort, and above all prioritize gaining strategic clarity over rushing into any particular fix.

The Mechanism — Philosophy With a Deadline

Bostrom's Fields Medal jab reframes what "important work" means under time pressure: a discovery's value isn't the value of the fact discovered, but the value of having it earlier than it would otherwise have arrived — so a brilliant mind spending a career solving a famously hard but not-urgent problem may have contributed less than it seems, if the problem would have been solved soon anyway by someone else. The same logic applies to philosophy itself: many of philosophy's eternal questions (metaphysics, the nature of value) are worth answering, but a superintelligence — or even just modestly enhanced humans — would answer them far better than we currently can. This licenses a strategy of deferred gratification: postpone the eternal questions, and instead spend scarce present-day brilliance on the one urgent, time-sensitive task — making sure we get competent successors at all.

The Shift — Selection Criteria for What to Work On

Three filters for choosing where to direct effort: urgency (needed before the intelligence explosion, not just eventually valuable — Bostrom flags that some AI-capability research may be net negative, since it speeds arrival without speeding the control-problem solutions needed to survive it), robust positive value (beneficial across a wide range of possible scenarios and defensible from a wide range of moral views, since strategic uncertainty means good intentions can still be counterproductive), and elasticity (does one more unit of effort actually move the needle — "encouraging more kindness in the world" fails this test despite being important and urgent, because there's no known lever that converts extra effort into more kindness at any usable rate). Two objectives survive all three filters: strategic analysis (the search for crucial considerations — ideas that could overturn not just implementation details but the entire direction of effort, since missing even one could make otherwise valiant work "as actively harmful as those of a soldier fighting on the wrong side") and capacity-building (assembling a well-resourced, truth-seeking support base early, before the field's culture calcifies around less careful incentives).

Key Episode — The Samurai and the Octopus

Bostrom's sharpest image for institutional integrity under pressure: imagine a project that has sunk years and millions into a promising but not-quite-safe AI design, and a crucial consideration emerges showing a different approach would be safer. Does the project "kill itself off like a dishonored samurai," abandoning sunk progress for safety? Or does it "react like a worried octopus, puffing out a cloud of motivated skepticism" to explain the inconvenient finding away? Building institutions and cultures that reliably choose the samurai's path — that can actually act on a discovered crucial consideration rather than rationalize around it — is, Bostrom argues, its own distinct and underappreciated challenge, separate from discovering the considerations in the first place.

The Shift — Children With a Bomb

The book's closing image: humanity resembles small children playing with an undetonated bomb — except there isn't one child but many, each with an independent trigger, no way to simply back out of the room (an intelligence explosion's consequences are global, not local), and no adult in the vicinity to take charge. Bostrom is explicit that this calls not for panic or fatalism but for a particular register: "a bitter determination to be as competent as we can," treating the challenge like an unavoidable exam that will either fulfill humanity's potential or foreclose it — while still holding onto ordinary human decency, common sense, and groundedness in the process, since the task, however civilizationally central, must still be carried out by people living ordinary lives in the meantime.

Key Terms

  • Deferred gratification (philosophical) — postponing answerable-later questions to focus scarce present effort on time-sensitive ones
  • Crucial consideration — an idea capable of overturning the general direction of effort, not just its implementation details
  • Elasticity (of a problem) — how much one additional unit of effort actually changes the outcome
  • Robust positive value — beneficial across a wide range of scenarios and moral views, given strategic uncertainty