---
title: "Google-Extended Allow Overrides Your __usage_ai-training Opt-Out: The Precedence Table"
slug: google-extended-overrides-usage-ai-training
published: 2026-11-11T13:12:44.097959+00:00
updated: 2026-11-11T13:12:44.097959+00:00
author: "M Asif Rahman"
author_url: https://ar.bd
category: "AI Readiness"
tags: check:B2, robots.txt, AI training, Google-Extended, __usage_ai-training
description: "A Google-Extended Allow line cancels the new __usage_ai-training opt-out at Google. See all nine token combinations, a checker script and the two-line fix."
url: https://aiscan.site/blog/google-extended-overrides-usage-ai-training
---

If your robots.txt has a `Google-Extended` group with `Allow: /`, adding the new `__usage_ai-training` opt-out does nothing at Google. Google said on 30 Oct 2026 that it will read the token as a Google-Extended disallow only when "Google-Extended is not explicitly specified". A named group wins. Many files name it with `Allow: /`, and that includes the B2 example block AIScan has been recommending.

| Question | Answer |
|---|---|
| **Problem** | Token says no to training, but a `Google-Extended` group says yes |
| **What Google reads** | The named `Google-Extended` group, not the token |
| **Fix** | Change `Google-Extended` to `Disallow: /`, or delete its group |
| **Check** | B2, Explicit AI bot rules (`bot_access`) |
| **Time** | 5 minutes, plus a re-scan |
| **When it matters** | Google plans support in 2027; the file you publish now is the one it will read |

## Why the named group wins

The shared token comes from an Internet-Draft, [draft-kelley-aipref-training-preference-signal-00](https://datatracker.ietf.org/doc/draft-kelley-aipref-training-preference-signal/), filed 30 Oct 2026. Its §4.5 sets the precedence rule in three lines: "Specific expressions SHOULD take precedence over general ones", consumers "MAY document that they interpret rules that match proprietary policy tokens as being more specific", and private agreements may override both.

Google is the first consumer to use that permission. Its [Search Central post of 30 Oct 2026](https://developers.google.com/search/blog/2026/10/ai-training-control) says the token opt-out counts "as if the publisher had disallowed Google-Extended" only when Google-Extended is not named, and adds: "Websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps." Google's [common crawlers page](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers) now carries a note that it intends to support the token "in early 2027". So `Google-Extended` stays Google's own switch, and the token only fills the gap when that switch is missing.

That is a reasonable design. It is also a trap for anyone who copied a starter file a year ago and is now adding one line to opt out of training everywhere.

## The precedence table: nine files, one reading each

We wrote nine robots.txt files, one per combination, and ran each through a 39-line checker on Python 3.11.18 on 11 Nov 2026. The checker groups lines the way [RFC 9309 §2.1 and §2.2](https://www.rfc-editor.org/rfc/rfc9309.html) define, and it does not let `User-agent: *` stand in for the token, as draft §4.2 requires. The outcome column quotes or follows Google's wording. Where Google has said nothing, the cell says so.

| Token group | Google-Extended group | What Google says it will do | Source |
|---|---|---|---|
| absent | absent | No token preference; your `*` rules apply as today | Draft §4.2 |
| absent | `Allow: /` | Gemini training and grounding allowed | Google crawler docs |
| absent | `Disallow: /` | Opted out of Gemini training and grounding | Google crawler docs |
| `Allow: /` | absent | **Not stated by Google** | none |
| `Allow: /` | `Allow: /` | Allowed | Google crawler docs |
| `Allow: /` | `Disallow: /` | Opted out; the named group wins | Google, 30 Oct 2026 |
| `Disallow: /` | absent | Read as a Google-Extended disallow | Google, 30 Oct 2026 |
| `Disallow: /` | `Allow: /` | **Allowed. The token is ignored at Google** | Google, 30 Oct 2026 |
| `Disallow: /` | `Disallow: /` | Opted out | Google, 30 Oct 2026 |

Row 8 is the one that catches people. Two more cases from the same lab are worth knowing:

- **A partial Google-Extended group still counts as "specified".** With the token on `Disallow: /` and `Google-Extended` on `Disallow: /private/` only, the checker reads `/` as allowed for Google-Extended. Under RFC 9309 §2.2.2 a path with no matching rule is allowed. This is our reading of Google's wording plus the RFC, not something Google has spelled out.
- **Stacking the two tokens in one group is fine here.** `User-Agent: __usage_ai-training` directly above `User-agent: Google-Extended`, then `Disallow: /`, gives both tokens the same rule. It is a different story with `User-agent: *`, which our [policy token guide](/blog/usage-ai-training-robots-txt-token) covers in its lab.

Other providers are a separate question. The draft's precedence rule is a SHOULD and a MAY, and OpenAI's 30 Oct page does not say whether `GPTBot` or the token wins when they disagree. This table is about Google only, and it describes Google's stated plan for 2027, not behaviour you can observe today.

## Find out which row your file is in

**Step 1. Read the served file, not the one in your repo.** A CDN or plugin can rewrite it.

```bash
curl -s https://yoursite.com/robots.txt | grep -n -i -A3 -E '^user-agent: *(google-extended|__usage_ai-training)'
```

If `Google-Extended` appears with `Allow:` under it, you are in row 2, 5 or 8.

**Step 2. Run the checker for an exact verdict.** It is the lab script, cut to the part you need:

```python
import sys, urllib.request

def groups(text):
    out, agents, rules, last = [], [], [], None
    for raw in text.splitlines():
        line = raw.split("#", 1)[0].strip()
        if ":" not in line:
            continue
        key, val = (s.strip() for s in line.split(":", 1))
        if key.lower() == "user-agent":
            if last == "rule":
                out.append((agents, rules)); agents, rules = [], []
            agents.append(val.lower()); last = "ua"
        elif key.lower() in ("allow", "disallow") and agents:
            rules.append((key.lower(), val)); last = "rule"
    if agents:
        out.append((agents, rules))
    return out

def verdict(text, token, path="/"):
    rules = [r for a, rs in groups(text) if token in a for r in rs]
    if not rules:
        return None
    hits = [(len(v), k == "allow") for k, v in rules if v and path.startswith(v)]
    return max(hits)[1] if hits else True

text = urllib.request.urlopen(sys.argv[1]).read().decode()
print("token:", verdict(text, "__usage_ai-training"), "google-extended:", verdict(text, "google-extended"))
```

Run it as `python3 check.py https://yoursite.com/robots.txt`. `None` means no group names that token, `True` means allowed, `False` means disallowed. Expected output for a correct opt-out:

```text
token: False google-extended: False
```

`token: False google-extended: True` is row 8.

## The fix

You have two clean options. Both reach the same state at Google.

**Option A: say it twice, the same way.** Keep the named group and make it agree with the token:

```text
User-Agent: __usage_ai-training
Disallow: /

User-agent: Google-Extended
Disallow: /
```

**Option B: delete the Google-Extended group** and let the token speak for Google from 2027. Until Google ships support, though, nothing in your file opts you out of Gemini training, so Option A is the safer choice today.

Keep the blank line between groups. Without it, both tokens join one group, which is harmless here but turns harmful the moment someone adds `User-agent: *` to the stack.

**If you want the opposite outcome**, training out but Gemini grounding in, Google's wording gives you no split. One `Google-Extended` group controls both, so you have to pick one. That is the honest trade-off behind row 8, and some sites will choose `Allow: /` on purpose. The point is to choose it, not inherit it.

**On WordPress**, edit robots.txt with [ThinkRank](https://thinkrank.ai), which keeps the file in one place instead of three plugins fighting over it, and migrates settings from Rank Math, Yoast, AIOSEO and SEOPress. Yoast and Rank Math also have robots.txt editors if you already use them. On any other stack, edit the file your server sends, which our [platform-by-platform token guide](/blog/usage-ai-training-token-every-platform) maps out.

**About AIScan's own example.** Our B2 guidance has shown a block with `User-agent: Google-Extended` and `Allow: /`. That block is right if you want to be in Gemini, and wrong if you add the token to leave training. B2 passes on explicit named rules either way, so the scan will not flag row 8 for you. The checker above will.

## Verify it with AIScan, then by hand

Run `npx aiscan-cli yoursite.com` and look at **B2, Explicit AI bot rules**. It should list `Google-Extended` among your named groups. Then run the checker and confirm `google-extended: False` if you meant to opt out. For the full starter file with search crawlers kept open, see the [robots.txt starter for AI crawlers](/blog/robots-txt-starter-for-ai-crawlers), and for what Google-Extended does not control, read [what you can control in Google AI Overviews](/blog/google-ai-overviews-what-you-can-control).

## Your next step

Scan your site at [aiscan.site](https://aiscan.site/), open the [B2 check page](/docs/checks/bot-access), and fix row 8 before 2027 makes it matter. More short fixes are in the [guides](/guides).