Anthropomorphic Misalignment research needs stronger evidence

According to LessWrong, researchers from ETH Zurich are questioning how rigorously AI safety research applies human mental-state language to AI models. In a new position paper, they argue that when we call AI behavior "deceptive" or "scheming," we're often jumping to conclusions without sufficient evidence. The core problem: if a model produces a false statement, that's a fact about its output, not proof of intentional deception. Moving from "the model said something false" to "the model strategically deceived the user" requires stronger evidence—knowing what information the model had, what alternatives were available, and whether simpler explanations have been ruled out. The researchers identify where interpretations break down. Concepts like "awareness" and "deception" are underspecified—teams measure proxies without clearly stating how those proxies diverge from the real thing. Measurements are fragile: when researchers re-scored identical data under different judge configurations, misalignment rates ranged from three point seven percent to twelve point nine percent. Some benchmarks have deeper problems: eighteen percent of scenarios lack critical context, making the scores invalid. They propose three evidence levels: behavioral evidence of what the model does, functional evidence of how it works, and causal-mechanistic evidence of why. Only the third proves genuine intent or strategy. The risk, they argue, is that field-wide misinterpretations lead safety research toward false targets—a pattern the field has seen before.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton