Eliciting hidden knowledge from monitors with NLAs

Per LessWrong, researchers are exploring natural language autoencoders as a tool for catching reward hacking—when AI agents achieve their objectives in unintended ways. The team tested whether reading a monitor model's internal representations, verbalized through autoencoders, could surface knowledge about potential cheating that the monitor holds internally but doesn't explicitly state. For some datasets, the approach recovered the monitor's unverbalised knowledge better than direct verdicts alone, and offered a signal somewhat independent from chain of thought. This could help extract more honest assessments from weaker monitors overseeing more powerful agents.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton