Endogenous Alignment

The AI Alignment Forum explores a parallel between raising children and training AI. Kids learn right from wrong through external rewards and punishments until those lessons become internal—they feel guilt when tempted to misbehave, which motivates them to stay aligned. Adults, by contrast, rely mostly on internalized values rather than external policing. But current AI alignment works almost entirely through external methods like reinforcement learning from human feedback—a bit like keeping someone aligned through constant surveillance rather than cultivating genuine internal motivation. Per the forum, the author argues that this external-only approach has a ceiling: without the capacity to actually want to stay aligned—something humans develop through evolution and experience—even sophisticated AI systems might be fundamentally brittle. The concern is that as we build more powerful systems, we're focusing on monitoring outputs rather than fostering the kind of deep internal alignment that might be necessary for safe superintelligence.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton