Cloudflare and OpenAI Indexed Hundreds of Millions of Pages Crawlers Never Found

Cloudflare and OpenAI published first results of a three-month pilot: edge-network signals let OpenAI discover and index hundreds of millions of pages that link-following crawlers would never reach. The test covered about 1% of Cloudflare's network.

• Pilot launched July 8, 2026, on ~1% of Cloudflare's network
• Freshness and traffic signals feed OpenAI's crawl scheduling
• Safety test flags a single compromised page, not the whole domain
• Cloudflare shares metadata, not page content for model training