Evolution of my AI Safety threat models

An AI safety researcher on LessWrong documented the evolution of their threat models over a year, moving from a scattered list to a structured framework for thinking about AI risks. The author initially compiled worries about misaligned AI, reward misalignment, economic disruption, and human misuse — but found the list disorganized. Over recent months, they introduced two axes for organizing these threats: whether they stem from AI systems themselves or from how humans might misuse technology, and whether concentrated or distributed agency drives the risk. Per the post, this framework surfaces new risk considerations — like how open-sourcing powerful AI creates different problems than centralized control.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton