OpenAI Will Honor a robots.txt Signal on Using Websites to Train AI

What happened

OpenAI announced that it will honor a shared signal in the robots.txt file. With it, whoever manages a website can communicate their preferences about how their content is used to train artificial intelligence models. The idea is for this signal to work with all AI providers that join the initiative, not just OpenAI.

The announcement doesn't detail the signal's syntax or which other companies are participating. Even so, the change points to a more standardized way of saying "yes" or "no" to AI using your content.

What this means for your business

Anyone who works with SEO already knows robots.txt, since it guides search engine crawlers. Now it also takes on a role in your relationship with AI. A few practical suggestions:

• Talk to whoever manages your website and check what your robots.txt currently allows or blocks.
• Think through your strategy before blocking. For many small businesses, showing up in AI assistant answers can bring visibility. Blocking everything may reduce that exposure.
• Protect what sets you apart. If you publish exclusive materials, such as courses, original research or paid content, it's worth considering specific restrictions.
• Keep up with updates. Since participation depends on each provider, keep an eye on which companies start honoring the signal.

The decision doesn't have to be permanent: the file can be adjusted as your goals change.

Original story: openai.com.