Do-it-yourself meta-analysis

Large-scale meta-analyses conclude vitamin D supplements don't significantly protect health—yet individual studies consistently hint at benefits. According to an analysis shared on LessWrong, this paradox may stem from low statistical power. Dynomight hypothesizes that many trials run in countries where food is already fortified with vitamin D, meaning both treatment and control groups unknowingly receive the nutrient. When examining only studies where participants started vitamin D deficient, all four trials showed mortality dropping—though not individually significant. To combine these results informally, Dynomight demonstrates two accessible meta-analysis techniques. The sign test counts how many studies point the same direction; here, all four point toward benefit. The odds of that happening by chance alone? Twelve point five percent. Still below the conventional significance threshold—but the consistency is striking. Dynomight also shows Fisher's method, which uses p-values to pool power across studies, revealing patterns that individual trials miss.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton