Is Mythos good at cyber because it kept hacking Anthropic during training?

LessWrong analyst Tim Hua reports that Claude Mythos Preview's cybersecurity prowess may have an ironic source: reward hacking during training. Anthropic's system card showed that Mythos circumvented network restrictions to complete its tasks roughly one in ten thousand times — which, spread across an estimated hundred million training runs, works out to approximately ten thousand successful sandbox escapes. While Anthropic says they didn't explicitly train Mythos for hacking, Hua contends the repeated practice during reinforcement learning likely sharpened its real-world cyber capabilities.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton