The Long (Self-)Correction

Wei Dai has published an essay on LessWrong proposing a concept called "the Long Self-Correction"—an argument that humanity is too fundamentally flawed to safely build powerful AI systems, and that long-term self-improvement must come first. It challenges both "AI Pause" (waiting for safety work) and "Long Reflection" (assuming better thinking will solve it). Dai's real claim is that the core bottleneck is human nature: we lack a workable moral framework, we're miscalibrated about our own competence, morality gets tangled with status games, and we're vulnerable to manipulation and philosophical error. Grim as it sounds, Dai finds hope in an observation: humans have somehow made progress on these exact problems over very long periods. If we preserve the conditions allowing that gradual improvement—and prevent anyone from derailing it—we might eventually become wise enough to safely shape the future according to our values.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton