According to a post on Less Wrong, the AI alignment challenge might stem from philosophy's oldest problem: the 'ought-is' gap. PeterL observes that science can explain why cutting a vein causes blood loss and brain dysfunction, but science itself can't say whether that's bad. To solve alignment, he proposes 'extended science'—a framework that adds 'preference' as a fundamental property of particles, not just physical necessity. Systems would then be described by both what they must do and what they prefer to do. If values emerge from preference rather than mechanical law alone, perhaps aligned AI becomes less paradoxical. The author frames this as exploratory thinking, not a complete solution.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton