Using DSPy to evaluate and improve Datasette Agent's SQL system prompts

Simon Willison used DSPy—a framework for optimizing language model prompts—to improve Datasette Agent, a tool that answers questions by executing read-only SQL queries. Using Claude Code with Claude Fable five, he identified several promising improvements. The most notable: the agent was guessing column names because its instructions discouraged calling describe_table. Including column names directly in the schema listing would eliminate these errors.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton