Approximate Natural Latents Have Exact Prices

Information can be shared between observers in surprisingly constrained ways. Imagine Alice and Bob watching the same room through cameras in opposite corners. They see correlated images because it's the same room, but each camera shows different angles and details. A "natural latent" is something they can both accurately infer from their own feed alone—what's actually in the room, independent of perspective. How good can these shared inferences be? According to LessWrong, researcher Haru proved something elegant: when observations are jointly Gaussian—a common case in information theory—there's an exact mathematical relationship governing approximation errors. Two sources of error are at play. One: how much of the shared concept one camera can't recover from watching alone. Two: how much of their agreement the inferred concept fails to explain. These two errors must sum to a specific value. It's not a loose bound—it's a perfect identity, as rigid as a conservation law. Even though perfect shared concepts don't exist in real, noisy systems, approximate ones follow laws as rigid as physics. It's theoretical, but it precisely characterizes what kinds of information can genuinely be common to multiple observers.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton