How to Measure Intelligence Beyond Human Scale?

Traditional intelligence tests break down when the test-taker outthinks the test-maker. Per LessWrong, as AI systems now exceed human capabilities, benchmarks designed for human-scale intelligence lose their power to discriminate. Elad Hazan's paper proposes adversarial psychometrics: instead of relying on expert-curated questions or external judges, AI systems generate questions specifically to expose gaps in each other's abilities. The framework, called SepaRank, works like this. One system proposes a question and commits to an answer. Others respond with their own answer and a confidence level. The proposer scores points for questions that create wide variance in solver responses, while solvers earn points for well-calibrated predictions. No human jury required. The approach scales as models improve, and historically, the ability to distinguish capabilities in others maps closely to general intelligence itself—suggesting this method may measure something genuinely meaningful about what we call intelligence.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton