A circuit prior in NN-bayes

LessWrong notes that Kaarel, together with Dmitry, presented evidence that even heavily over‑parameterized neural networks trained in a Bayesian way exhibit a circuit prior, meaning that when the target function can be expressed by a modest circuit, only a modest amount of data is needed to achieve strong test performance, subject to certain scaling conditions and caveats. The talk’s slides, particularly slides thirty‑six and thirty‑seven, also outline a set of open problems for future neural‑network learning theory.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton