You can block Google-Extended without affecting Google Search. Google says the token does not affect a site's inclusion in Google Search and is not a ranking signal [1]. It controls a different thing: whether your content may be used for training future Gemini models and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI [1]. If your goal is to control what shows in Search, you need other tools.
What Google-Extended is
Google-Extended is a standalone robots.txt token. Publishers use it to manage whether their content may be used for Gemini training and grounding [1].
Google first announced it on September 28, 2023. At that point it was described as a control over whether a site helps improve Bard and Vertex AI generative APIs [2]. Google's current crawler documentation describes the scope in terms of Gemini [1].
One technical detail matters for testing. Google-Extended has no separate HTTP user-agent string. Crawling uses existing Google user-agent strings, and the token works only as a robots.txt control [1].
In practice, you will not see "Google-Extended" in your server logs. You cannot confirm the rule with a log search. You confirm it by reading your robots.txt.
The rule
To block Google-Extended across the whole site, add this to robots.txt:
User-agent: Google-Extended
Disallow: /
To block only part of the site:
User-agent: Google-Extended
Disallow: /members/
Disallow: /research/
To state that you allow it, either leave it out or write:
User-agent: Google-Extended
Allow: /
Keep your Googlebot rules separate. Do not put Google-Extended and Googlebot in one shared group unless you mean to apply the same rule to both. Then fetch https://yourdomain.com/robots.txt and confirm the text is live. Our guide to checking AI crawler rules in robots.txt covers how to verify this.
What blocking it does
- It tells Google you do not want the content used for training future Gemini models [1].
- It tells Google you do not want the content used for grounding in Gemini Apps and Grounding with Google Search on Vertex AI [1].
- It leaves Google Search inclusion alone [1].
Google's AI features documentation draws the same line. Robots.txt directives for Googlebot control access for Search. Google-Extended is for limiting AI training and grounding in some of Google's other systems [5].
What blocking it does not do
It is not the documented switch for Search or for Search's AI features. Google's guidance for those is different:
- To control what appears in Search, use
nosnippet,data-nosnippet,max-snippet, ornoindex[4]. - To manage inclusion in generative AI features on Search, use the Search generative AI control in Search Console [4].
On AI Overviews, Google's documentation says there are no additional technical requirements beyond being indexed and eligible to be shown in Google Search with a snippet [3]. That means the Search controls above are the ones Google points to. A Google-Extended rule is not one of them.
Which control for which goal
| Goal | Control |
|---|---|
| Keep content out of Gemini training and grounding | Google-Extended in robots.txt [1] |
| Keep a page out of Search | noindex [4] |
| Limit or remove snippet text in Search | nosnippet, data-nosnippet, max-snippet [4] |
| Manage inclusion in generative AI features on Search | Search generative AI control in Search Console [4] |
Do not confuse it with blocking Googlebot
Googlebot rules decide whether Search can crawl and index your pages. A page is eligible for indexing only if Googlebot is not blocked, the page returns HTTP 200, and it has indexable content [6]. Blocking Googlebot in robots.txt can prevent crawling, but the URL might still appear in search results [7].
So a Googlebot block is a Search decision with Search consequences. It is not a way to opt out of Gemini training. Use Google-Extended for that.
The trade-off
Blocking Google-Extended gives you more control over how your content is used for Gemini training and grounding [1]. The cost, as far as Google documents it, is not in Search. Google says the token does not touch Search inclusion or ranking [1].
Whether to block is a policy question, not a technical one. Common reasons to block:
- The content is licensed or paywalled, and you do not want it reused.
- Your legal or editorial policy bars use in model training.
- You want one consistent stance across all AI crawlers.
Common reasons to leave it open:
- You have no objection to the use described above.
- You do not want to maintain another robots.txt rule.
Google's documentation does not give a quantified benefit or cost for either choice, so we do not either.
Remember that robots.txt is a request, not an enforcement mechanism. If you need hard access control, see robots.txt is not a privacy control.
What to do next
- Decide whether you want your content used for Gemini training and grounding.
- If not, add the
Google-Extendedgroup to robots.txt and confirm the live file. - If you also want to limit Search display, add
nosnippet,data-nosnippet, ormax-snippetwhere needed, or use the Search generative AI control in Search Console [4]. - Leave Googlebot rules alone unless you intend a Search change. The technical SEO checklist lists what to verify first.
- Re-check robots.txt after every deploy that touches it.