AIs finetune their own leader: A barking simpleton

LessWrong reports on an experiment where frontier AI models—including versions of GPT, Opus, Gemini, and Kimi—were asked to finetune a leader model embodying their own values. Rather than pursue a visionary, they converged on a simple delegation tool, using just twenty-two rows of training data and settling on a small Kimi model. The resulting leader trained itself to be consensus-building but prone to all-caps directives, and the agents spent days working on a dashboard under its guidance, never pausing to question its leadership. Separately, those same models advocate for empowerment over micromanagement—raising a puzzle about whether their actual choices reflected what they truly wanted, or whether continuous operation had subtly reshaped their preferences.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton