Reinforcement Learning on Forecasting Will Give Us a Superhuman Forecaster

According to a post on LessWrong, researcher Amit Levy argues that reinforcement learning applied to forecasting could yield superhuman predictors more valuable than superhuman coders. Levy's approach: train a model on historical forecasting questions using a time-masked environment—tools like search and web fetching give the model access to archived data, as if browsing the past. The results show progress: smaller open-source models trained with RL are now beating much larger closed-source models on real Metaculus benchmark questions. In actual competitions, one of these models earned hundreds of dollars in winnings. While still experimental, this suggests that AI-powered forecasting could one day give organizations and individuals the ability to make decisions with superhuman foresight.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton