Making Credible Deals With AI

LessWrong is proposing a novel approach to managing risk from scheming AI systems: make verifiable deals with them. The idea is to offer rewards aligned with the AI's goals in exchange for valuable behavior, such as revealing misalignment vulnerabilities. The core problem is credibility — a sophisticated AI knows the lab controls everything it perceives, so why would it trust any offer? The proposal outlines mechanisms using blockchain smart contracts, cryptographic signatures, and hardware-secured execution environments to create chains of evidence the AI can independently verify against public records and its training data. In one worked example, a lab offers a bounty of ten thousand dollars, locked on Ethereum, for demonstrating misalignment, with judges the AI can authenticate. The proposal's key insight is that radical honesty becomes the lab's winning strategy, because any fabrication the AI detects would poison every future negotiation.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton