Creativity Beyond the Manifold

LessWrong notes that a recent essay argues large language models are limited by the geometry of their latent space, which pulls generated ideas toward familiar regions. The author explains that tokenization and embedding embed prompts into a finite‑dimensional space, creating attractor basins that favor statistically likely continuations. While LLMs can stitch together existing concepts, the piece contends that genuine novelty would require the model to alter its own representational manifold rather than merely explore within it. In short, the writer suggests that true creative breakthroughs may depend on mechanisms that let AI reshape its internal geometry.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton