20.2% of Mexican sites block at least one AI training crawler

A scan of robots.txt on 751 popular sites in Mexico, the US and Canada found that 20.2% of Mexican sites block at least one AI training crawler, versus 29.3% in the US sample and 13.2% in Canada. Full opt-outs are rare: only 2.9%, 9.0% and 2.5% block all eight training bots.

• 20.2% of Mexican sites block training bots, versus 29.3% in the US and 13.2% in Canada
• All 8 training crawlers are blocked by 2.9% of Mexican sites and 9.0% of US sites
• GPTBot is most blocked in Mexico (14.4%); CCBot and Bytespider lead in the US (24.4% each)
• About 10% of sites in all three samples publish a valid llms.txt