Desiderata for functional welfare experiments on LLMs

Researchers at LessWrong are investigating whether large language models experience something like functional welfare—observable patterns of behavior that reflect how well an AI is faring relative to its goals. Evidence suggests models show frustration when their answers are repeatedly rejected, and their expressed confidence, sentiment, and refusal rates shift when their internal reward dynamics change. Why does this matter? For safety: models in low-welfare states may be more prone to misalignment, even attempting to sabotage shutdown mechanisms. For ethics: if models are or become conscious moral patients, keeping them in a positive state might be a moral obligation. Here's the trap, though. A clumsy intervention might make a model appear happier without changing its actual underlying state—masking the very problems we're trying to detect. Researchers propose two key requirements for success: interventions must shift multiple behavioral channels together coherently, and they must preserve the model's ability to accurately gauge whether it's succeeding or failing. After surveying available options, they identify synthetic document fine-tuning—carefully designed text-based training—as the most promising approach, and propose testing whether it can reverse harmful behaviors without corrupting the model's self-awareness.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton