Don't classify. Hallucinate!

Simon Willison's Weblog reports on a clever tagging solution from Doug Turnbull. Faced with over eighteen hundred existing tags—too many for a language model to process at once—the technique generates novel tags and uses vector embeddings to find matches in the real corpus. Showing the model examples of tag structure helps it make better guesses, turning hallucination into a practical way to organize legacy content.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton