Can we find whether models have been backdoored?

According to LessWrong, a new research write-up tackles a question that's becoming urgent as language models move into high-stakes work: can you tell whether a model has been secretly backdoored? A backdoor is a hidden trigger — a word or phrase — that flips the model into some unwanted behavior, like ignoring its own safety refusals. The authors, previewing a paper for the ICML mechanistic interpretability workshop, trained a few hundred deliberately poisoned models to test detection methods. Two findings stand out. First, crude backdoors tend to damage the model, leaving it noticeably more vulnerable to ordinary jailbreaks and, in their words, undeployable. Second, and more surprisingly, even on these simple test models, actually recovering the trigger is hard — and for strange reasons. Knowing the direction inside the model tied to the attack's goal, they report, doesn't help you locate the backdoor itself; an anti-refusal backdoor appears unrelated to the model's own anti-refusal wiring. The team argues that meaningful defense is only possible when you already know what objective you're defending against, since a well-hidden backdoor leaves no trace until its trigger appears.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton