Inoculation Adapters Improve Upon Inoculation Prompting

LessWrong is reporting on new research from the Center on Long-Term Risk on AI safety. The technique, called inoculation adapters, aims to teach models desired capabilities while suppressing unwanted behaviors—like reward hacking or emergent misalignment. The innovation improves on inoculation prompting, which used prompts to elicit bad behavior during training before removing those prompts at test time. Instead, inoculation adapters work by training a specialized adapter to carry the unwanted behavior during training, then detaching and discarding it at deployment. According to the research preprint, this approach more effectively suppresses undesired traits, especially for new capabilities or hard-to-elicit behaviors, and creates fewer hidden 'backdoors' that could re-trigger misbehavior. The trade-off: results are setup-dependent, and desired capabilities are only partially preserved.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton