Fix guide cover: do noai meta tags stop AI crawlers? OpenAI, Anthropic and Google chips with a failing-to-passing window pair
Fix guide cover: do noai meta tags stop AI crawlers? OpenAI, Anthropic and Google chips with a failing-to-passing window pair
AI Readiness

Do noai and noimageai Meta Tags Stop AI Crawlers? What Each Vendor's Docs Say (2026)

No crawler operator documents noai or noimageai. We read eight vendors' crawler docs on 12 Nov 2026: the table, a checker script and the robots.txt fix.

12 Nov 2026 7 min read
#noai#robots meta#AI training#robots.txt

This guide covers B1 · Bot Access, B2 · Bot Access.

Table of contents

No. On 12 Nov 2026 we read the crawler documentation of eight AI operators (OpenAI, Anthropic, Google, Apple, Perplexity, Common Crawl, Amazon and Microsoft) and none of them documents noai or noimageai. The tag states a preference that no documented crawler acts on. Google's own robots meta page goes further: it supports two names in that tag, googlebot and googlebot-news, and "other values are ignored", so <meta name="google-extended" content="noai"> does nothing at Google.

QuestionAnswer
Does noai stop AI crawlers?No operator we checked documents it (8 of 8, read 12 Nov 2026)
Does noimageai?Same: 0 of 8
<meta name="google-extended">?Ignored. Google-Extended is a robots.txt token only
What does work todayA named User-agent group in robots.txt for each training crawler
The one page-level AI tag with a vendor behind itBing's ai-training:disallow, honoured from 1 Feb 2027
AIScan checkNone reads these tags. B1 (Content Signals) and B2 (named AI bot rules) read robots.txt
Time to fix10 minutes, plus a re-scan

Why the tag looks like it should work

noai borrows the shape of a real control. <meta name="robots" content="noindex"> works because search engines wrote it into their documentation and their crawlers parse it. noai and noimageai were added to the same tag by artist portfolio sites in the image-model rush, then copied into themes, plugins and forum answers. Nobody who runs a crawler signed up to read them. Threads like the r/artstation one asking "When you put NoAI on your online portfolio does it really..." keep asking the same question because the answer is in no vendor's docs.

The google-extended variant is a category error. Google's common crawlers page says Google-Extended "doesn't have a separate HTTP request user agent string" and that "the robots.txt user-agent token is used in a control capacity". It lives in robots.txt, not in a meta tag. Google's robots meta tag documentation names the two tokens it accepts in name= and ignores the rest.

What each operator documents (read 12 Nov 2026)

We fetched each page with a browser user agent on 12 Nov 2026 and searched it for noai, noimageai, robots.txt and page-level tags.

OperatorDocuments noai / noimageai?Documented AI controlPage-level option they do document
OpenAINoUser-agent: GPTBot in robots.txt ("Disallowing GPTBot indicates a site's content should not be used in training")None
AnthropicNoUser-agent: ClaudeBot in robots.txtNone
GoogleNoUser-agent: Google-Extended in robots.txtnosnippet keeps a page out of AI Overviews and AI Mode as direct input (Search, not training)
AppleNoUser-agent: Applebot-Extended in robots.txt for model trainingnosnippet opts content out of Siri and Search "broad world knowledge answers"
PerplexityNoUser-agent: PerplexityBot (search; Perplexity says it is not used for foundation models)None
Common CrawlNoUser-agent: CCBot / Disallow: /None
AmazonNoUser-agent: Amazonbot in robots.txtnoarchive, which Amazon defines as "do not use the page for model training"
Microsoft (Bing)NoUser-Agent: __usage_ai-training group, from 1 Feb 2027<meta name="Bingbot" content="ai-training:disallow"> and ai-grounding:, from 1 Feb 2027; NoArchive / NoCache until 1 Mar 2027

Two findings stand out. First, the page-level controls that do exist are spelled differently at every vendor: Amazon reuses noarchive, Apple and Google reuse nosnippet, and Bing is adding a new Bingbot vocabulary. Second, every one of the eight documents a robots.txt route. That is the only control they share.

Bing's post of 30 Oct 2026 also says its new preferences "will replace existing Microsoft preferences NoArchive and NoCache", and that after 1 March 2027 those two "will no longer be read as a preference for training or grounding". If you already use noarchive for Bing, plan that move now. Our guide to the __usage_ai-training token covers the robots.txt half.

Find the tag on your own pages

This one line lists any noai tag in a page's served HTML:

curl -s https://yoursite.com/ | grep -ioE '<meta[^>]+content="[^"]*no(image)?ai[^"]*"'

Headers can carry it too, so check curl -sI https://yoursite.com/ | grep -i x-robots-tag.

For a fuller reading we wrote a 34-line Python checker. It reads meta tags and X-Robots-Tag headers and prints, for each AI-related rule, which operator documents it. We ran it on Python 3.11.18 on 12 Nov 2026 against a local test page carrying every variant, and against our own four sites. Save it as ai-meta-check.py and pass it one or more URLs:

import re, sys, urllib.request

# Which operator documents each page-level rule (vendor docs read 12 Nov 2026)
DOCS = {
    "ai-training": "Microsoft Bingbot, from 1 Feb 2027",
    "ai-grounding": "Microsoft Bingbot, from 1 Feb 2027",
    "noarchive": "Amazon (no model training); Bing until 1 Mar 2027",
    "nocache": "Bing until 1 Mar 2027",
    "nosnippet": "Apple (no world-knowledge answers); Google (no AI Overviews input)",
}
AGENTS = {"robots", "googlebot", "bingbot", "google-extended", "gptbot", "claudebot"}

def rules(url):
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
    with urllib.request.urlopen(req, timeout=20) as r:
        html = r.read().decode("utf8", "ignore")
        for h in r.headers.get_all("X-Robots-Tag") or []:
            m = re.match(r"\s*([\w-]+):\s*(.*)", h)
            name, val = (m.groups() if m and m.group(1).lower() in AGENTS else ("robots", h))
            yield "header", name.lower(), val
    for tag in re.findall(r"<meta\b[^>]*>", html, re.I):
        n = re.search(r'name=["\']?([\w-]+)', tag, re.I)
        c = re.search(r'content=["\']([^"\']*)', tag, re.I)
        if n and c and n.group(1).lower() in AGENTS:
            yield "meta", n.group(1).lower(), c.group(1)

for url in sys.argv[1:]:
    print(url)
    for src, name, val in rules(url):
        for rule in (v.strip().lower() for v in val.split(",")):
            key = rule.split(":")[0]
            if key in ("noai", "noimageai") or key in DOCS or name == "google-extended":
                doc = None if name == "google-extended" else DOCS.get(key)
                print(f"  {src:6} {name}: {rule:22} -> {doc or 'no operator documents this'}")

The output:

http://127.0.0.1:8765/test.html
  header robots: noai                   -> no operator documents this
  header robots: noimageai              -> no operator documents this
  header bingbot: ai-grounding:disallowextended -> Microsoft Bingbot, from 1 Feb 2027
  meta   robots: noai                   -> no operator documents this
  meta   robots: noimageai              -> no operator documents this
  meta   google-extended: noai                   -> no operator documents this
  meta   bingbot: ai-training:disallow   -> Microsoft Bingbot, from 1 Feb 2027
  meta   robots: noarchive              -> Amazon (no model training); Bing until 1 Mar 2027
https://aiscan.site/
https://xspeedcache.com/
https://storeseo.com/
https://bettershield.ai/

None of our four sites carries a page-level AI tag. Their training preferences live in robots.txt, which is where the table above says they belong.

The fix: say it where crawlers read it

  1. Decide which operators you want to keep out of training. Leave search crawlers (Googlebot, Bingbot, OAI-SearchBot, PerplexityBot) open unless you want to leave those search products too.
  2. Add one named group per training crawler to robots.txt, each with its own User-agent line and a blank line between groups:
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: CCBot
Disallow: /

User-Agent: __usage_ai-training
Disallow: /
  1. Check for a Google-Extended group with Allow: / elsewhere in the file. It wins over the shared token at Google, as our precedence table shows.
  2. For Bing from 1 Feb 2027, add <meta name="Bingbot" content="ai-training:disallow"> on pages that need a page-level rule, or rely on the token group above for the whole site.
  3. Declare a policy as well, if you like. Cloudflare's Content-Signal line (what AIScan's B1 reads) and the other declaration formats are compared in our content-use policy guide. The robots.txt starter for AI crawlers has a complete file.

On WordPress, edit robots.txt and robots meta from one place with ThinkRank. One plugin owns the file instead of three fighting over it, and it migrates settings from Rank Math, Yoast, AIOSEO and SEOPress. Yoast and Rank Math also have file editors if you already run one of them.

Should you delete the noai tag? No need. It costs nothing, breaks nothing, and some sites keep it as a public statement. Treat it as a stance, not a control, and do not let it stand in for the robots.txt groups.

Verify it with AIScan, then by hand

Run npx aiscan-cli yoursite.com and open B2, Explicit AI bot rules on the bot access check page. It should list GPTBot, ClaudeBot and the other names you added. B1 should show your Content-Signal line if you added one. AIScan does not score noai either way, because nothing reads it.

By hand: curl -s https://yoursite.com/robots.txt should print each group, and the file should return 200 with text/plain.

Your next step

Scan your site at aiscan.site, check B2, and move any noai-only opt-out into robots.txt this week. More short fixes are in the guides.

Frequently asked questions

Do noai and noimageai meta tags stop AI crawlers?

Not by any documented rule. On 12 Nov 2026 none of eight operators (OpenAI, Anthropic, Google, Apple, Perplexity, Common Crawl, Amazon, Microsoft) documented either value. Use named robots.txt groups such as GPTBot, ClaudeBot and Google-Extended instead.

I added <meta name="google-extended" content="noai"> but my site is still used by Gemini. Why?

Google's robots meta documentation accepts only googlebot and googlebot-news as names and ignores other values. Google-Extended is a robots.txt token. Add a User-agent: Google-Extended group with Disallow: / to robots.txt.

My robots meta says noai, noimageai, but GPTBot still appears in my server logs. Is the tag broken?

The tag is not broken; OpenAI never said it reads it. OpenAI documents GPTBot control through robots.txt. Add User-agent: GPTBot with Disallow: / and allow a day for the change to be picked up.

An AI image generator produced work in my style even though my portfolio uses noimageai. What went wrong?

The tag is a request no image-model crawler we checked documents, and older crawls may predate it. Robots.txt groups for the training crawlers, such as CCBot and Google-Extended, are the documented route, and they only affect future crawls.

Is there any page-level tag that opts out of AI training?

Two operators document one. Amazon says its crawlers read noarchive as do not use the page for model training. Bing will honour <meta name="Bingbot" content="ai-training:disallow"> from 1 Feb 2027, and reads NoArchive and NoCache until 1 Mar 2027.

Does AIScan flag a missing or present noai tag?

No. No AIScan check reads it, because no crawler documents it. B1 reads the Content-Signal line in robots.txt and B2 reads named AI bot groups. A noai tag neither raises nor lowers your score.

Should I remove noai from my theme?

There is no need. It costs nothing and breaks nothing, and some sites keep it as a public stance. Just do not rely on it: put the real opt-out in robots.txt.

Does nosnippet keep my content out of AI answers?

Partly, and only at two operators. Google says nosnippet stops the page being used as direct input for AI Overviews and AI Mode, and Apple says it opts content out of its broad world knowledge answers. It also removes your search snippet, so weigh that cost first.

M Asif Rahman

About the author

Entrepreneur, curious about AI

I have been building companies for a long time, and lately most of my curiosity goes to AI. AIScan is a small personal project I built on a flight from Dubai to Narita. I use it to understand how ready the web is for AI agents, and I share what the scans teach me.

Related guides

Browse every guide by check and platform