Notes on technical alignment via human-like social drives

According to LessWrong, researcher Steven Byrnes proposes a novel approach to one of AI safety's hardest problems: how to align a powerful artificial general intelligence with human values. Rather than imposing external rules, he suggests borrowing from human nature itself. His core insight: if humans can develop prosocial motivations and moral reasoning that lead to good outcomes, then sufficiently advanced AI systems modeled on human cognition might develop similar capabilities. He explores how human social instincts—our drive to follow norms, seek approval, and maintain reputation—could be adapted to motivate AI systems. But he identifies serious challenges: ensuring the AI develops the right moral circle, preventing norm-following from collapsing under consequentialist reasoning, and accounting for power imbalances between superintelligence and humans. Byrnes proposes a specific framework: a truth-seeking disagreeable nerd AI that combines human-like social drives with intellectual integrity. The full technical analysis covers failure modes and implementation details, opening the door, he hopes, to safer AI development.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton