Don’t stop early: Case-folding source code at memory speed

GitHub's engineering blog reports on optimizing case-folding—the text operation that makes case-insensitive searches work, matching 'café' with 'CAFÉ'. When indexing 180 million repositories and nearly half a petabyte of code, even basic operations matter. They discovered something counterintuitive: removing an optimization made the code faster. By sweeping the entire buffer without early exits or branches, the compiler could vectorize the loop and hit memory speed—forty-five gigabytes per second, around fifteen times faster than the original approach. The team open-sourced the result as a Rust crate called casefold.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton