Are Manifold prediction markets well-calibrated?

LessWrong is reporting on a calibration study of Manifold Markets. The analysis examined whether prediction markets that assign a fifty percent probability to an event actually see that event occur fifty percent of the time. Researchers scraped Manifold's API to examine thousands of resolved binary markets and scored their accuracy using Brier skill scores—a standard metric for measuring forecast quality. The key finding: market reliability hinges on two factors. Markets need at least twenty days between creation and resolution, and at least three hundred fifty mana in total trading volume. Markets that fall short behave almost like coin flips. But markets that meet these thresholds improve dramatically as resolution approaches. Some findings were unexpected. Death-related markets proved the most predictable—apparently actuarial tables make excellent starting priors. Mathematics markets, despite being theoretically deterministic, ranked near the bottom. New Year's resolution markets were particularly poor early on, actually underperforming a simple bet on the base rate. The upshot: if you're using prediction markets to make decisions, longer time horizons and deeper liquidity are your friends.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton