the polysemanticity of polysemanticity in language models

According to LessWrong, researchers exploring how language models work from the inside have identified a crucial phenomenon called polysemanticity. It's what happens when a neural network doesn't have enough neurons to represent all the different concepts it encounters. Instead of giving each concept its own neuron, the network learns to reuse neurons—the same neuron fires for multiple completely different inputs. Think of it like a single light switch controlling multiple rooms simultaneously. The model figures out which room was actually meant based on which combination of switches flipped. This matters deeply for AI safety researchers trying to understand model behavior. When they probe individual neurons, they might miss how concepts are actually encoded across clusters of neurons working together.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton