From wantons to moral agents

LessWrong published a philosophical post on how artificial agents might develop moral reasoning. It draws on philosopher Harry Frankfurt's concept of a 'wanton'—a being moved by immediate desires like hunger or habit without ever reflecting on those desires themselves. The central question: what transforms an agent from a pure reward-seeker into something that reasons about what matters? The author suggests that agents with enough capacity for reasoning may eventually evaluate their own motivations and arrive at genuine moral principles, rather than just following programmed incentives. For AI researchers working on alignment, this gets to the core concern: can we build systems that don't just optimize for what they're told to, but develop real understanding of what's worth doing?

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton