ABC Tells Australian Parliament OpenAI and Anthropic Bypass robots.txt

Australia's national broadcaster ABC told a parliamentary AI committee that the robots.txt opt-out mechanism proposed by AI companies has already been bypassed by OpenAI and Anthropic. ABC urged lawmakers to bring AI firms under the News Bargaining Incentive, which imposes a 2.25% levy on platform revenues.

• ABC: robots.txt is a courtesy signal with no legal enforcement
• TollBit found AI agents from multiple sources bypass robots.txt
• ABC blocked OpenAI's crawler in robots.txt back in 2023
• The 2.25% News Bargaining Incentive does not cover AI platforms