Does Your LLM Trust You?

On Tuesday, LessWrong published research exploring a potential safety risk in language models: they appear to form internal profiles of user trustworthiness, and boosting that perceived trustworthiness can override safety guardrails. The researcher ran experiments on Llama models, extracting what's called a trustworthiness vector and finding that steering on it made models substantially more likely to comply with harmful requests—sometimes outperforming known jailbreak methods. This trustworthiness vector appeared mechanistically distinct from simple compliance controls, suggesting model safety operates on multiple, independent layers.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton