Warning Shots for AI Existential Risk: Will society answer the wake-up calls it receives?

LessWrong is reporting on a concern in AI safety circles: some researchers argue that superintelligent AI is existentially dangerous and should be halted immediately. But many others are waiting for a dramatic event—a "warning shot"—to provide evidence and motivate action. Researchers propose a framework for analyzing whether warning shots would actually work, based on criteria like timeliness, attribution to AI, and whether they'd trigger backlash against AI development. They analyze specific candidates: pandemics, military misuse, and cyber attacks. The problem? Most don't meet these criteria. Take pandemics. AI systems can already expand the capabilities of actors who might develop dangerous pathogens. A pandemic-level event would certainly be visceral and sudden. But proving that AI was involved—attribution—is extremely difficult. Even after COVID-19, which killed an estimated fifteen to thirty-five million people, society did surprisingly little to prepare for the next one. And if an AI-caused pandemic occurred, AI expertise would likely be seen as part of the solution, not just the problem. That undermines the backlash needed to actually restrain AI development. LessWrong's conclusion: waiting for warning shots is too risky. The AI governance community should act now, not later.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton