Your Brain Has an Attack Surface part 2

Per LessWrong, researcher Igor Pereverzev is studying how hidden messages can move through neural networks without detection. He examined SpikeGPT, a spiking neural network model, asking whether covert signals relocate to a new position or collapse entirely. Initially, he used a ratio—shift to deformation—of nine point seven to one as proof that signals had moved. But when he repeated the test, the ratio changed: six point seven in one run, seven point two in another, making the metric unreliable. More critically, the ratio was fundamentally flawed: two opposite scenarios—hidden classes staying intact versus merging completely—could produce the same shift and deformation values. The metric couldn't tell them apart. So Pereverzev devised a real test: retrain a detection classifier from scratch on the moved signals. One hundred percent recovery means the signal relocated; failure to recover means it collapsed. On SpikeGPT, recovery was complete across all attempts, proving relocation works.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton