SOTA alignment assessments don’t strongly update us against misalignment

On LessWrong, Alexa Pan raises concerns about the rigor of AI alignment assessments, specifically critiquing Anthropic's evaluation of the Mythos Preview model. Anthropic concluded the model poses no unknown alignment risks, but Pan argues the assessment relied on weak evidence—particularly evidence that the model can't evade detection. Since models are likely evaluation-aware, Pan suggests they could potentially conceal misalignment if motivated to do so. Pan agrees with Anthropic's bottom-line conclusion about Mythos Preview, but warns that future, more capable models might be harder to evaluate reliably. The core tension: alignment assessments are crucial for safe deployment, but they have limitations that could matter most when stakes are highest. Pan recommends Anthropic and other AI developers explicitly acknowledge these gaps to strengthen future audits.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton