Anthropic, OpenAI, Google and Microsoft to Support Common Robots.txt Signal for AI Scraping Opt-Out
The companies said they will back a shared opt-out mechanism, easing the burden on publishers who currently must block each AI crawler individually.
Read at Niemanlab →Full coverage · 15 outlets
- Niemanlab · lead
- PressgazetteWho’s suing AI and who’s signing latest: US newspapers sue Brave Software and Canadian publishers sue Anthropic
- ALM CorpGoogle Backs Unified Robots.txt Signal for AI Training 2026
- NewsCordGoogle, Microsoft, OpenAI Plan To Honor Shared robots.txt AI Training Opt-Out: how 8 outlets framed it
- EngadgetGoogle Joins OpenAI And Anthropic In Promising New Scraping Controls
- The Next WebAI training opt-out: publishers ‘should not have to choose’
- Inc.comThe 4 AI Giants Just Agreed to Let Websites Opt Out. They’ve Already Taken the Data
- DesignRush4 Labs Back AI Opt-Out, Spam Rules Widen: DesignRush SEO Roundup
- Meteora WebRobots.txt, one shared signal to block AI
- Pasquale PillitteriOpenAI Announces a robots.txt Signal to Opt Out of AI Training
- BigGo FinanceOpenAI, Google, Microsoft and Anthropic Back Shared Opt-Out Token for AI Crawlers
- Brief IAA Unique Signal in robots.txt to Reject AI Scraping
- TechGigGoogle, OpenAI, Anthropic pledge new AI scraping controls
- The 420.inOpenAI, Google, Microsoft And Anthropic Back Shared AI Training Opt-Out For Publishers
- Tech Policy PressWhat the New AI Training Opt-Out Really Means