What would it take for AI to discover penicillin?

The promise of autonomous labs is that AI can systematize discovery—running experiments, analyzing results, learning from failures. But according to a recent LessWrong essay, there's a fundamental problem: an AI lab optimizing bacterial yield might accidentally produce a powerful antibiotic that kills all the bacteria—showing zero-percent yield—and discard it entirely. Unlike Alexander Fleming's serendipitous discovery of penicillin, the breakthrough gets deleted. LessWrong contributor Connor Blake identifies two core limitations. First, autonomous labs optimize for a specific numerical signal, but Goodhart's Law applies: what you measure isn't what you want. A genuine discovery might show up as a failure or a glitch. Second, and perhaps more critical, the solution usually lies outside your current toolkit. Science's greatest advances often come from inventing entirely new tools. Freeman Dyson notes that over five hundred years, there have been roughly twenty tool-driven scientific revolutions compared to just six concept-driven ones. Graphene was discovered with Scotch tape. The implication is stark: if the next breakthrough requires a tool that hasn't been invented yet, no algorithm will find it in your autonomous lab. A true AI scientist, Blake suggests, would need human-level dexterity and reasoning to invent custom instruments on the fly. Today's autonomous labs are sophisticated optimizers—not scientists.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton