Tracking AI progress across 18 cognitive dimensions (ADeLe scales)

Researchers have introduced a new framework called ADeLe to track AI progress across eighteen different cognitive dimensions, such as spatio-physical reasoning and graduate-level knowledge. As LessWrong reports, the system uses a non-saturating scale to avoid the common problem of benchmarks becoming too easy as models improve. Analysis of twenty-one large language models shows consistent, linear progress across all observed dimensions, with most models surpassing the top twenty percent of thirteen-to-fourteen-year-old students between twenty twenty-three and twenty twenty-four. The researchers note that while domain knowledge often correlates with model size, reasoning and abstraction are more heavily boosted by inference-heavy models.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton