Google and the shared AI training control in robots.txt
Google plans to support a single robots.txt signal to say whether your content can be used to train AI. What it is, what it does not change and what to check on your site today.
A client asks whether your website “lets AI copy everything.” You open your robots.txt, see three blocks of rules someone pasted months ago and have no idea which one does what. On October 30, Google announced something that will simplify that conversation, even though it is not active yet.
Google plans to support a single robots.txt signal to say whether your content can be used to train AI. Here is what is known, what is not and what to do in the meantime.
What exactly did Google announce?
It announced that it plans to support a shared AI training control inside robots.txt. According to the announcement on Google Search Central, the idea is that a site uses a single signal, based on the work of the IETF AIPREF group, instead of writing different rules for each AI provider.
Google says it aims to support it, along with other providers, during 2027, and that it will post updates on progress. Today it is a plan, not a feature you can turn on.
What would the signal look like in your file?
It would be a normal robots.txt group whose agent name is a policy token. The draft published on the IETF Datatracker includes this example: it blocks training across the whole site except the press folder.
User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/
The draft clarifies that it adds no new directives: robots.txt paths and conflict resolution do not change. It is also clear about its status: it is an individual draft, it is not endorsed by the IETF and it has no official standing yet. Google's documentation mentions that same token and refers to early 2027.
What happens to Google-Extended?
Google-Extended stays the same and coexists with the new signal. It is the token that, since 2023, lets you limit whether the content Google crawls is used to train future Gemini models. Google explains that it has no user agent of its own and that it does not affect inclusion in Google Search or serve as a ranking signal.
On how they combine, the announcement says that if you use the shared signal to deny training and do not name Google-Extended, Google will interpret it as if you had blocked it. If you want a different rule for Google, you write it with Google-Extended.
| Question | What the official sources say |
|---|---|
| Does the shared signal work yet? | No: Google plans to support it in 2027 |
| Is it an approved standard? | No: it is an individual IETF draft |
| Does Google-Extended stop working? | No: it keeps working alongside the new signal |
| Does it change your position in Google Search? | No: Google-Extended is not a ranking signal |
| Do you need to change anything today? | It is not mandatory |
Should I block training or allow it?
It depends on what you want to gain: control over your content or more visibility in AI answers. Google's announcement notes that sites that specifically allow Google-Extended can appear in answers and receive clicks through grounding in the Gemini apps.
If your content is your product (courses, reports, a news archive), blocking training can make sense. If you live off being found, an open stance can help you. That decision is a business one, not a technical one, and the best basis for making it is in our guide to robots.txt and AI bots.
Heads up: training and search are different things. A rule meant to stop training, if poorly written, can take you out of places where you do want to appear. Change one rule at a time and check the result.
What can I do today without waiting for 2027?
Review your robots.txt, write down the decision you want to make and leave the file tidy. Three short steps:
- Open your-site.com/robots.txt and read each block of rules. If you do not know why one exists, make a note of it.
- Define your stance: allow, block or mix depending on your site's folders.
- Document the decision so that when the shared signal arrives you only have to translate it.
If your site is on Cloudflare, go over what the Cloudflare free plan includes before touching anything. And if what you want is to appear in AI answers, the step-by-step GEO guide and the llms.txt guide are the natural complement.
In short
When someone asks whether your site “lets AI copy it,” you will be able to answer with one clear decision. Google plans to support a shared robots.txt signal during 2027, Google-Extended remains in force and none of this affects your position in Google Search. If you want your robots.txt, your GEO and your stance on AI sorted out with good judgment, check out our SEO and GEO service to appear in AI.
Frequently asked questions
What did Google announce about robots.txt and AI?
On October 30, 2026, Google said it plans to support a shared robots.txt signal, based on the work of the IETF AIPREF group, so a site can state its preference on AI training.
When will it start working?
Google says it aims to support it along with other AI providers in 2027. Google's documentation mentions early 2027.
Does it replace Google-Extended?
No. Google says each company's own controls will keep working alongside the new shared signal.
Does it affect my position in Google Search?
According to Google's documentation, Google-Extended does not affect inclusion in Google Search and is not used as a ranking signal.
Do I have to change my robots.txt today?
It is not mandatory. The signal is still a draft and Google will support it later. What helps is reviewing which AI rules you have and why.
Do not stop at the theory: apply it to your business
We implement SEO and GEO with AI end to end, with your real operation. The first conversation is free.
