According to LessWrong, researchers published findings from a survey of AI safety experts on continual learning in language model agents. Continual learning is when an AI system updates and learns new capabilities during deployment without catastrophically forgetting what it already knows. The survey found little consensus on the timeline. Five of seven respondents believed widespread continual learning agents are unlikely in the next three years. But four of six thought the first transformative AI systems—those capable of performing most human jobs—will be continual learners. All respondents agreed on two key points: continual learning would increase the attack surface for adversarial attacks, and more research into it would benefit AI safety. They also largely expected continual learning agents to reflect on their own goals, though only about half worried they'd undergo significant goal drift as a result. The researchers note their sequence has already evolved beyond what survey participants commented on, leaving some questions about whether the skepticism would extend to their latest proposals.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton