Kimi K3, and what we can still learn from the pelican benchmark

Chinese AI lab Moonshot announced Kimi K3 on Thursday, describing it as their most capable model to date, with two point eight trillion parameters. The model is available via API, with an open-weight release promised by late July. Moonshot's benchmarks show K3 outperforming Claude Opus and GPT-5.5, though lagging newer releases of both. Pricing is three dollars per million input tokens and fifteen dollars per million output tokens—competitive with Anthropic's Claude Sonnet series. According to Simon Willison's Weblog, Willison tested the model with his quirky 'pelican benchmark'—asking it to generate an SVG image of a pelican riding a bicycle. He notes the test has become less useful for model comparisons, but still values it as a 'hello world' for trying a new system. The real challenge today, he argues, is building models that can reliably use tools and maintain coherence over long conversations—and a drawing of a pelican doesn't assess any of that.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton