How do neural networks regress cubic polynomials? Apparently, they use a trick invented in Milan 500 years ago

According to LessWrong, researchers at the Wolfram Summer Research Institute found that neural networks learning to regress cubic polynomials independently rediscover mathematical techniques from centuries ago. Using interpretability tools called Sparse Autoencoders to peer inside the trained network, they found that it develops internal features resembling a step from the Cardano Method—a mathematical transformation for solving cubic equations developed in sixteenth-century Milan. Rather than memorizing, the network learns abstract features that mirror elegant mathematical tools humans discovered over centuries of calculus and problem-solving. The finding raises a striking question: when solving similar mathematical problems, do neural networks naturally converge on the same mathematical abstractions that mathematicians developed?

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton