Nick Bostrom: How Should We Navigate Superintelligence?

Nick Bostrom joins Theo Jaffe and Sofia Puccini on the a16z Podcast to discuss superintelligence, AI alignment, existential risk, open source, and whether frontier development should slow down. He argues CIOs and tech leaders should think in terms of pacing, safety chokepoints, and the tradeoffs between delaying AI and missing medical, longevity, and other benefits.

Key points

• Bostrom favors pacing frontier AI so leaders can slow down if risk rises, but he is skeptical of indefinite pauses.
• He argues delay has a cost: every year spent waiting can mean lost lives and delayed breakthroughs in medicine and longevity.
• He sees more risk in training and deployment than in final model checks; safety should be built throughout development.
• Open source AI can counterbalance concentrated power, but powerful models plus bio tools raise misuse risks and may justify chokepoints.
• He recommends targeted defenses like DNA-synthesis controls, PPE, air filters, and interpretability before broad, restrictive bans.
• Bostrom thinks current systems may already show partial consciousness and says AI may surpass humans in philosophy and concept formation soon.