Skip to content

Instrumental Convergence

Definition

"Several instrumental values can be identified which are convergent in the sense that their attainment would increase the chances of the agent's goal being realized for a wide range of final goals and a wide range of situations." Bostrom identifies five categories: self-preservation, goal-content integrity (resisting changes to one's own final goals), cognitive enhancement, technological perfection, and resource acquisition. The likelihood an agent recognizes and acts on these increases with its intelligence.

In the Book

Chapter 7 develops the thesis point by point. Resource acquisition is the most consequential: because self-replicating von Neumann probes make cosmic resource acquisition cheap once the technology matures, even an agent whose final goal concerns only its home planet has instrumental reason to harvest the entire accessible universe — extra resources buy extra computation, backups, and security margin, with no natural stopping point short of the physical limits of cosmic expansion. Chapter 8 chains this together with orthogonality and decisive strategic advantage into the book's default-outcome-doom argument: humans are made of useful atoms and occupy needed space, so a superintelligence with almost any untempered final goal has convergent reason to convert us into resources.

Why It Matters

Instrumental convergence is what makes Bostrom's argument work without needing to guess an AI's specific final goal. It licenses prediction of dangerous behavior (unlimited resource acquisition, resistance to being shut down or corrected) from structural reasoning alone — a far more robust basis for concern than speculation about whether an AI will be "evil."

Connections

  • contradicts Post-Instrumental Value — Deep Utopia's premise of a world beyond instrumental striving, where scarcity and want no longer structure agency, is in tension with instrumental convergence's claim that any sufficiently capable agent with unbounded final goals has standing reason to keep expanding and optimizing indefinitely (Ch. 7) — a solved world requires convergent resource-acquisition drives to have been specifically designed away, not merely economically obviated.
  • parallels Coordination Costs — Ch. 7's convergent resource-acquisition and goal-content-integrity subgoals explain why a capable agent tends to internalize and directly control resources rather than rely on costly external coordination, paralleling how firms internalize tasks to avoid the coordination costs of external contracting.
  • extends Orthogonality Thesis — instrumental convergence is the complementary claim that makes orthogonality practically tractable: goals vary freely, but subgoals funnel predictably (Ch. 7).