Running a 28.9M parameter LLM on an $8 microcontroller

An engineer has built a twenty-eight point nine million parameter language model that runs on an eight-dollar ESP32-S3 microcontroller, according to Adafruit Blog—roughly a hundred times larger than the previous model-on-chip record. The trick is a memory-management approach called Per-Layer Embeddings, adapted from Google's Gemma: the model's massive embedding table, twenty-five million parameters, stays in slow flash storage instead of fast RAM, and the system pulls only the rows it needs for each token, about four hundred fifty bytes at a time. Meanwhile, the reasoning core—the part that does actual computation—lives in fast memory where it can work efficiently. Trained on TinyStories, the model spins out short narratives and keeps them mostly coherent, though it can't answer questions, write code, or know facts. The project is MIT licensed and available on GitHub.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton