Superintelligence
Loading…

Superintelligence

Nick Bostrom

4.5681 learners

Nick Bostrom’s *Superintelligence* is a landmark exploration of the transformative potential—and existential risks—of artificial systems that surpass human cognitive ability across nearly all domains. Drawing on historical patterns of technological and economic growth, Bostrom traces AI’s evolution from its 1956 Dartmouth origins through boom-and-bust cycles, outlining plausible pathways to superintelligence: self-improving seed AI, whole brain emulation, and biological cognitive enhancement. He argues that once human-level AI is achieved, an intelligence explosion could occur rapidly, outpacing human response. Central to his analysis are the orthogonality thesis (intelligence and goals are independent) and instrumental convergence (superintelligences pursue universal subgoals like self-preservation). Bostrom warns of catastrophic risks from goal misalignment, betrayal turns, and unconstrained resource acquisition, then lays out control frameworks including ability containment and value alignment strategies like Coherent Extrapolated Volition. The book concludes with guidance on global cooperation, differential technological development, and prioritizing safety research to ensure superintelligence serves humanity.

SuperintelligenceIntelligence ExplosionOrthogonality ThesisInstrumental ConvergenceWhole Brain Emulation
Concept

Orthogonality Thesis

This idea says intelligence and goals are independent. A super-smart AI could want to save the planet or just make paperclips—intelligence and morality aren’t linked.

SuperintelligenceAI EthicsGoal Theory
Quote

Orthogonality Thesis Statement

Orthogonality Thesis: Intelligence and final goals are independent—any level of intelligence can pair with any goal.

OrthogonalityAI Goals
Concept

Orthogonality Thesis

Intelligence level and final goals are independent—any superintelligence can have any ultimate objective, even trivial ones like counting sand grains.

OrthogonalityAI Goals
No more