You can block Google-Extended without affecting Google Search. Google says the token does not affect a site's inclusion in Google Search and is not a ranking signal [1]. It controls a different thing: whether your content may be used for training future Gemini models and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI [1]. If your goal is to control what shows in Search, you need other tools.

What Google-Extended is

Google-Extended is a standalone robots.txt token. Publishers use it to manage whether their content may be used for Gemini training and grounding [1].

Google first announced it on September 28, 2023. At that point it was described as a control over whether a site helps improve Bard and Vertex AI generative APIs [2]. Google's current crawler documentation describes the scope in terms of Gemini [1].

One technical detail matters for testing. Google-Extended has no separate HTTP user-agent string. Crawling uses existing Google user-agent strings, and the token works only as a robots.txt control [1].

In practice, you will not see "Google-Extended" in your server logs. You cannot confirm the rule with a log search. You confirm it by reading your robots.txt.

The rule

To block Google-Extended across the whole site, add this to robots.txt:

User-agent: Google-Extended
Disallow: /

To block only part of the site:

User-agent: Google-Extended
Disallow: /members/
Disallow: /research/

To state that you allow it, either leave it out or write:

User-agent: Google-Extended
Allow: /

Keep your Googlebot rules separate. Do not put Google-Extended and Googlebot in one shared group unless you mean to apply the same rule to both. Then fetch https://yourdomain.com/robots.txt and confirm the text is live. Our guide to checking AI crawler rules in robots.txt covers how to verify this.

What blocking it does

  • It tells Google you do not want the content used for training future Gemini models [1].
  • It tells Google you do not want the content used for grounding in Gemini Apps and Grounding with Google Search on Vertex AI [1].
  • It leaves Google Search inclusion alone [1].

Google's AI features documentation draws the same line. Robots.txt directives for Googlebot control access for Search. Google-Extended is for limiting AI training and grounding in some of Google's other systems [5].

What blocking it does not do

It is not the documented switch for Search or for Search's AI features. Google's guidance for those is different:

  • To control what appears in Search, use nosnippet, data-nosnippet, max-snippet, or noindex [4].
  • To manage inclusion in generative AI features on Search, use the Search generative AI control in Search Console [4].

On AI Overviews, Google's documentation says there are no additional technical requirements beyond being indexed and eligible to be shown in Google Search with a snippet [3]. That means the Search controls above are the ones Google points to. A Google-Extended rule is not one of them.

Which control for which goal

GoalControl
Keep content out of Gemini training and groundingGoogle-Extended in robots.txt [1]
Keep a page out of Searchnoindex [4]
Limit or remove snippet text in Searchnosnippet, data-nosnippet, max-snippet [4]
Manage inclusion in generative AI features on SearchSearch generative AI control in Search Console [4]

Do not confuse it with blocking Googlebot

Googlebot rules decide whether Search can crawl and index your pages. A page is eligible for indexing only if Googlebot is not blocked, the page returns HTTP 200, and it has indexable content [6]. Blocking Googlebot in robots.txt can prevent crawling, but the URL might still appear in search results [7].

So a Googlebot block is a Search decision with Search consequences. It is not a way to opt out of Gemini training. Use Google-Extended for that.

The trade-off

Blocking Google-Extended gives you more control over how your content is used for Gemini training and grounding [1]. The cost, as far as Google documents it, is not in Search. Google says the token does not touch Search inclusion or ranking [1].

Whether to block is a policy question, not a technical one. Common reasons to block:

  • The content is licensed or paywalled, and you do not want it reused.
  • Your legal or editorial policy bars use in model training.
  • You want one consistent stance across all AI crawlers.

Common reasons to leave it open:

  • You have no objection to the use described above.
  • You do not want to maintain another robots.txt rule.

Google's documentation does not give a quantified benefit or cost for either choice, so we do not either.

Remember that robots.txt is a request, not an enforcement mechanism. If you need hard access control, see robots.txt is not a privacy control.

What to do next

  1. Decide whether you want your content used for Gemini training and grounding.
  2. If not, add the Google-Extended group to robots.txt and confirm the live file.
  3. If you also want to limit Search display, add nosnippet, data-nosnippet, or max-snippet where needed, or use the Search generative AI control in Search Console [4].
  4. Leave Googlebot rules alone unless you intend a Search change. The technical SEO checklist lists what to verify first.
  5. Re-check robots.txt after every deploy that touches it.