Cloudflare splits AI crawler controls into Search, Agent and Training

Cloudflare replaced its one-click AI bot blocking with three separate categories: Search, Agent and Training. Search is allowed by default for new domains, while Training and Agent crawlers are blocked by default on ad-supported pages. Because multi-purpose crawlers like Googlebot combine search and training behavior, blocking Training can also affect search indexing.

• New system deployed on September 15, 2026, replacing the 'Block AI Bots' option
• Search is allowed by default for new domains
• Training and Agent crawlers blocked by default on pages with ads
• Blocking Training can affect Googlebot due to combined crawler behavior