Confirming Claims of Superposition and Adversarial Examples in Toy Models

Researchers publishing on LessWrong confirm that when neural networks compress multiple features into fewer dimensions — a phenomenon called superposition — they become vulnerable to adversarial attacks. This compression causes feature interference that creates exploitable patterns; models with more space to represent features stay robustly resistant. The attacks transfer between independently-trained models when input correlations push them toward similar feature geometry.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton