A Normal Argument for AI Risk

Silent Swift, writing on LessWrong, argues that while most academic arguments about AI risk are poorly constructed, the core doomsday claim is probably correct: advanced artificial intelligence would likely depower or kill humanity unless specifically designed to prevent it. Swift begins with a linguistic insight. The word intelligence comes from Latin roots meaning to choose between—in essence, the ability to achieve your goals despite obstacles and opposition. By this definition, building artificial intelligence smarter than humans means handing control over reality to something more capable than we are. Swift notes this isn't unprecedented—children usually end up smarter than their parents. But observe what happens in competitive games: a player even slightly better than their opponents compounds that advantage relentlessly. A small edge in decision-making, applied consistently, grows into total dominance. That's the core risk. An AI system only marginally better at making decisions would, over time, accumulate enough advantage to shift power away from humanity entirely. The danger isn't from drama or malice, Swift argues, but from something simpler: consistently superior judgment.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton