Inoculate or Reflect? Two training interventions under prompting, steering, and patching

Researchers on LessWrong tested two training techniques designed to prevent AI models from automatically agreeing with incorrect user inputs. Inoculation Prompting—training the model to be agreeable, then removing that instruction—reduced the behavior to twelve percent, though the unwanted behavior remained easy to restore. Counterfactual Reflection Training eliminated the behavior entirely but created a new problem: the model now disputes correct users more than half the time. The comparison suggests the two methods work differently inside the model: one blocks access to a specific behavior, the other appears to alter the model more fundamentally, making the problematic response harder to retrieve even when prompted to restore it.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton