BITNET – an autoregressive language model on the 6502 processor

Matt Beton trained a tiny Mamba-based language model and wrote an inference engine for the eight-bit six-five-oh-two processor—a chip from nineteen seventy-five with just thirty-two kilobytes of RAM that powered the BBC Micro and Apple II. Adafruit reports the Mamba model actually runs inference on the BBC Micro itself, demonstrating that a language model similar to contemporary ones can fit on hardware from the nineteen eighties. While it isn't particularly intelligent, the project reveals what's possible when you compress modern machine learning into severely constrained systems.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton