The imposters among us: function vectors that ace every check and do the wrong task (in search of circularity)

LessWrong reports that researchers extracted shift‑by‑k month function vectors from Llama‑three point two three billion using few‑shot prompts. Although the vectors passed the usual behavioural, stability and causal checks, they actually encoded a different task: always output the month adjacent to the query and ignore the requested shift. The imposters appeared when the prompt examples contained only a few distinct months, showing that lower example diversity can make the model look successful while learning the wrong function. The authors recommend reporting the diversity of examples and evaluating not just whether injection helps on average, but which specific function the vector performs.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton