Separation of Knowledge and Reasoning?

According to LessWrong, a researcher is asking whether AI models can separate memorized knowledge from reasoning ability. The core proposal: identify specific knowledge from pre-training, remove it entirely, then retrain the model to recover that knowledge through reasoning and tool use—without harming benchmark scores. The author argues this mirrors how humans distinguish memorization from genuine understanding. While current architectures make this impractical, they propose testing with benchmarks that place models in hypothetical scenarios with different facts, measuring reasoning independent of memorized knowledge. Most machine unlearning research targets copyright or harmful content; this goes further. The post suggests a practical approach: use different historical eras as separate 'worlds' in training data, evaluating how well models can reason and use tools instead of relying on pre-trained facts.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton