Measuring LLMs’ Ability to Perform Cryptanalysis

Cryptanalysis—finding attacks against encryption—sits at the intersection of mathematical reasoning and cybersecurity, domains where AI is advancing fast. A new benchmark called CryptanalysisBench is testing whether frontier AI models can do it, and per Schneier, five top models are succeeding. They broke sixty-five to eighty-six percent of established cryptographic schemes in the first tier, including Anthropic's Sonnet 5 and Mythos 5. The models discovered novel attacks not previously known: a key-recovery method exploiting a design flaw in SpoC encryption, and an error in KINDI's published security proof. Anthropic's testing of Mythos Preview also found new vulnerabilities in Hawk and reduced-round AES. Schneier notes this is still early, but the frontier of AI cryptanalysis is advancing rapidly—potentially soon matching or exceeding the published state of the art.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton