Value Generalisation 2: The Missing Hole in AIs’ abilities

Stuart Armstrong argues in a LessWrong post that large language models have stalled — they feel perpetually on the verge of artificial general intelligence, but never arrive there. The missing piece, according to Armstrong, is what he calls strong generalization: the human ability to adapt to novel situations, plan long-term, and keep your bearings when conditions change. LLMs excel at remixing training data and passing benchmarks, but they fail to truly generalize their knowledge to reality, which explains their paradoxical mix of intelligence and glaring limitations. For AI to be reliably aligned with human values, Armstrong argues, it will need to develop explicit strong value generalization — not just more data or clever prompting tricks.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton