AI will make biological extinction risks worse before it makes them better

According to a LessWrong post, a common AI safety argument has a critical flaw. The argument goes: we should build superintelligence fast to protect ourselves from extinction risks like engineered bioweapons. But this logic backfires. AI will make biological extinction risks worse before making them better. Here's why: as AI accelerates scientific progress broadly, humanity's knowledge of biology advances—bringing us closer to being able to create extinction-level pathogens, even if AI itself refuses to assist. Speeding up a century of research progress into a decade brings forward the timeline for when such knowledge becomes possible. There's a deeper paradox. The only way superintelligent AI could truly prevent bioweapon development is by controlling everything. If humans retain any autonomy, they can still misuse advanced biological knowledge. This creates a brutal tradeoff. Minimizing biological extinction risk from AI acceleration requires a very short window between 'AI speeds up science' and 'AI takes over'—leaving little time to solve the alignment problem itself. The conclusion: accelerating AI development is not a viable strategy for reducing extinction risk.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton