A new transparency audit by Google DeepMind's interpretability team examined DiffusionGemma, the company's text diffusion model—a different architecture from traditional language models. According to the research published on LessWrong, DiffusionGemma is about as interpretable as the standard Gemma model, despite working quite differently under the hood. The key twist: diffusion models generate all tokens simultaneously, while autoregressive models build text one word at a time. That parallel approach makes it harder to understand why the model made specific choices. The team found they could apply techniques like the logit lens to decode what's happening at intermediate steps, but reconstructing the full reasoning process—the algorithm itself—remains a challenge. Interestingly, the model sometimes makes mistakes early on and corrects them retroactively as it refines the full text. The research matters for AI safety, since future models may perform even more reasoning in hidden, latent spaces. The audit provides a template for evaluating transparency in those next-generation architectures.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton