Cloudflare Separates AI Training Controls From Search Indexing for Website Owners

Cloudflare introduced a Disallow AI Training setting that prohibits the use of site content for training AI models without blocking search indexing. The directive is published in robots.txt via Bot Preference Sync; Applebot and Googlebot already comply, Bingbot from early 2027.

• Setting distinguishes crawlers by three signals: Search, Training, and Agent
• Applebot and Googlebot will continue indexing sites but not training models on them
• Bingbot will support the directive in early 2027
• Old Block AI Bots is being disabled, clients migrated to new settings