[Macroagents] 2. Design lenses for optimizing macroagents

LessWrong's latest post builds on its macroagent ontology, introducing five design lenses for shaping large AI systems: incentives, evolution, information routing, knowledge aggregation, and value aggregation. The author highlights two failure modes when tasks are hard to measure — slop, where agents slack off on ungraded work, and goodharting, where they chase proxies instead of the real goal. The post also argues that bad incentive structures often persist as stable equilibria, using a signaling equilibrium example to show how costly credentials can lock themselves in. The takeaway is that improving macroagents means tackling these structural issues, not just boosting individual competence.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton