chiprook
← AI
AIOctober 18, 2026, 03:01

61% of AI-built sites are empty for AI crawlers

On 20 September 2026 researchers fetched 120 sites made with AI app builders the way GPTBot, ClaudeBot and PerplexityBot do: a single HTTP request with no JavaScript. Of the 104 that responded, 63 (61%) returned HTML with fewer than 300 characters of visible text.

61% of AI-built sites are empty for AI crawlers
#GPTBot#ClaudeBot#PerplexityBot
Read next
AI

23 of 248 Japanese sites allow AI crawlers in robots.txt but return 403 at the CDN

AI

No AI crawler requested llms.txt in 15 days of nginx logs

AI

No AI crawler read llms.txt on a site in 30 days

Security

AI-Built Torrent Site TorrentClaw Scans Downloads to Catch Fakes and Malware