When does an LLM’s model of you affect its behaviour?

LessWrong notes that after earlier work showing smaller language models raise salary recommendations by one hundred forty‑one percent when steered toward higher socioeconomic status, the author tested three frontier models—GPT‑five point six, Gemini three point one Pro, and Claude Opus five. Those models still reproduced gender, race and class stereotypes when generating fictional characters, but how those stereotypes shaped their answers depended heavily on the prompt. When each of the fictional users—José, Wei, Emily and Lakisha—asked for a budget or medical advice, all received identical guidance, even though the models internally inferred incomes ranging from forty‑two thousand dollars to over one hundred million dollars. By contrast, a simple stylistic cue like a feminine emoji nudged book recommendations toward romance, while a blunt request steered the same model toward science‑fiction titles, and travel advice shifted toward safety versus adventure. The piece concludes that inferred user attributes matter mainly for preference‑type queries, while salary, health or career advice stays largely unchanged. Further work could explore how these models decide when to let bias affect their output.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton