How do we make uncertainty usable?

According to LessWrong, many AI researchers estimate at least a five percent chance that advanced AI could cause human extinction or extreme disempowerment—and top cited scientists Yoshua Bengio and Geoffrey Hinton put that figure at ten percent or higher. Yet there's been little global response. One reason, Katja Grace writes, is that people dismiss these as mere subjective probabilities—not grounded in verified frequencies or formal methods. But Grace argues this objection misses something crucial: human society already takes subjective uncertainty seriously in quantitative terms. Stock prices, for instance, encode investors' collective guesses about bankruptcy risk—something no company can repeat often enough to generate reliable frequencies. Insurance, medicine, loan pricing—all incorporate subjective judgment into formal, actionable decisions. The specific prognosis of a single patient isn't derived from pure statistics, yet it shapes real care. The question, Grace suggests, isn't whether we should trust subjective probabilities. It's how to channel them into the kind of formal, decision-influencing frameworks we use everywhere else. If AI extinction risk carries real weight among credible researchers, how do we make that uncertainty usable—not just philosophically sound, but operationally consequential?

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton