According to LessWrong, AI safety researchers are grappling with an uncomfortable possibility: their efforts to prevent catastrophic AI outcomes might actually make things worse. The concern isn't new. Holden Karnofsky, a prominent figure in AI governance, recently suggested the odds of AI safety work being net-negative are roughly even—or worse. LessWrong contributor Elias Schmied compiled a list of specific ways safety efforts could backfire: strict AI regulation might spark geopolitical conflict or empower authoritarians. Activist campaigns could polarize people against the cause entirely. Attempts to control powerful AI might provoke adversarial behavior. Safety research could accidentally accelerate AI development by training a new generation of capable researchers. And if future AIs are conscious beings, preventing their development might itself be a moral wrong. The post isn't claiming AI safety has been net-negative. Rather, it argues the field suffers from overconfidence about its own impact.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton