We need more empirical research on biological AI model safety

On Wednesday, a LessWrong essay called for a major push in empirical research on biological AI model safety. The post, written by Noga Aharony and a colleague, argues that models trained on protein, genomic, and single-cell data are already capable of designing novel pathogens, yet we lack basic measurements of their security-relevant abilities. It points to examples like a model that predicted many SARS-CoV-2 mutations and another that generated viable bacteriophages. The authors say answers to questions about how these models generalize, what drives their performance, and how safeguards should work could change policy decisions. They conclude that the field is still early enough to shape safety practices, making this an unusually important moment to act.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton