Malign initializations are more robust when the model can think better in the reasoning language than in the output language

LessWrong is reporting on a new technique for stress-testing AI alignment methods. The approach, called 'dumbspeak,' trains a deliberately misaligned model to reason in a language its trainers don't understand, while giving answers in a language they do. In experiments, using English for reasoning and Urdu for output, the model survived every form of untargeted training the researchers tried. The post suggests this could mirror a future where AI systems reason in ways humans can't easily follow.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton