When does a chess transformer “see” a knight fork? An initial result from logit lens and attention patterns

According to LessWrong, researchers are using mechanistic interpretability—a technique for reverse-engineering AI models—to understand how a chess-playing transformer learns to recognize tactics. The model, called Maia 3, is specifically designed to imitate human play at various skill levels. By analyzing the residual stream of the network at each layer using a technique called logit lens, the researchers found strong evidence that the neural representation of knight forks—a chess tactic where a single knight move attacks two valuable pieces—snaps into focus after the model's fifth attention block. This suggests that tactical understanding doesn't emerge gradually but crystallizes at a specific depth in the network. The findings are part of a larger research program exploring how genuine chess skills form as the model's skill level increases, with implications for understanding how artificial intelligence develops conceptual knowledge.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton