Say it once: introducing Bot Preference SynC

Cloudflare is introducing a new feature called Bot Preference Sync to help website owners manage how AI crawlers interact with their content. As Cloudflare tells it, the tool automatically updates a site's robots dot text file to match the AI bot configurations set in the company's dashboard, ensuring that stated preferences and enforced blocks are consistent. This allows site owners to more easily distinguish between traffic intended for search indexing, AI agents, and model training. The feature is available to all customers, from the free tier to enterprise, and will be enabled by default for new users.

Listen to this story

Hear this and more stories in a personalized audio briefing.

Open The Chonkerton