According to LessWrong, decision theory has a hidden assumption: the decision-maker isn't part of the world being decided about. The author explains why this matters by comparing two approaches: the "petty system," where you ignore the decider and focus narrowly on what's modeled—like a quantum chemist ignoring herself when calculating molecular properties—and the "grand system," where everything, including the decider, is part of the world. In everyday applications, this distinction doesn't matter. But Newcomb's problem, a classic thought experiment, forces the issue. If a superintelligence can predict your choice because you're already embedded in the causal structure, then your decision actually determines what's in the boxes before you choose. Classical decision theory, which assumes you stand outside the world you're deciding about, fails here. The upshot: decision systems that account for the decider being inside the universe, rather than pretending they stand outside it, may be essential for solving real-world reasoning problems.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton