---
title: "How to declare allow/deny rules for named AI crawlers in robots.txt on Wix"
slug: robots-txt-ai-bots-wix
published: 2026-09-07T13:23:46.042982+00:00
updated: 2026-09-07T13:23:46.042982+00:00
author: "Asif Rahman"
author_url: https://masifrahman.com
category: "AI Readiness"
tags: check:B2, check:D1, platform:wix, robots.txt, AI crawlers, Wix, bot access
description: "Wix generates your robots.txt. Here is how to add allow and deny rules for named AI crawlers in the Robots.txt Editor, and how to check the rules went live."
url: https://aiscan.site/blog/robots-txt-ai-bots-wix
---

Wix writes your robots.txt for you. Fetched on 7 September 2026, the file at `www.wix.com/robots.txt` ran to 90 non-empty lines: one `User-agent: *` group, 87 `Disallow` rules, and a single `Sitemap:` line at the bottom. Adding a rule for GPTBot or ClaudeBot takes about two minutes in the Robots.txt Editor, and it does something most guides never mention: the crawler you just named stops reading every one of those 87 lines. That trade is the whole of this job on Wix.

## Quick summary

| If you want to | Do this on Wix | Where it lives | Time |
|---|---|---|---|
| Stop AI training crawlers | Add named groups for `GPTBot`, `ClaudeBot`, `CCBot`, `Google-Extended` | **SEO & GEO > Tools and settings > Robots.txt Editor** | 2 min |
| Stay citable in ChatGPT and Perplexity | Leave `OAI-SearchBot` and `PerplexityBot` out of your deny list | Same editor | 0 min |
| Undo a bad edit | **Reset to Default**, then **Reset** | Same editor | 1 min |
| Keep one page out of AI Overviews | Tick **nosnippet** | Editor > **Pages & Menu > SEO basics > Advanced SEO** | 1 min |
| Confirm the rules are live | `npx aiscan-cli yoursite.com`, read rows **D1** and **B2** | aiscan.site | 1 min |

Wix names four crawlers in its own worked example. One of the four appears in no operator token list we hold.

## Wix keeps rewriting the file you are about to edit

According to Wix's article *Editing Your Site's Robots.txt File*, read on 7 September 2026, the platform maintains the file continuously: *"If you need to update a page, Wix automatically updates your robots.txt after you publish the page. If you change your site's settings, your robots.txt file is updated immediately."*

That is the opposite of how the other generated file on Wix behaves. As covered in [publishing a valid llms.txt on Wix](https://aiscan.site/blog/llms-txt-wix), Wix says that once you edit llms.txt, *"it will stop updating automatically so your changes are preserved."* Robots.txt carries no such lock: your named groups sit alongside a `*` group Wix goes on regenerating around them. Wix also puts a warning above the procedure worth repeating: *"Wix Customer Care cannot help you with any changes you make to your robots.txt file."*

## Open the Robots.txt Editor

Wix documents one path for reading, editing and resetting:

1. Go to **SEO & GEO** in your site's dashboard.
2. Scroll down to **Tools and settings**.
3. Click **Robots.txt Editor**.
4. Click **View File**.
5. Type your directives under **This is your current file**.
6. Click **Save Changes**, then **Save**.

To undo everything, the same four clicks then **Reset to Default** and **Reset**. There is no file manager and no FTP on Wix, so this panel is the only way in.

## A named group replaces the defaults, it does not extend them

RFC 9309 section 2.2.1 states verbatim: *"Crawlers MUST use case-insensitive matching to find the group that matches the product token and then obey the rules of the group."* Groups for different tokens do not stack; the spec merges only groups matching the same token.

Every rule Wix generated sits under `User-agent: *`. The moment you add `User-agent: GPTBot`, that crawler reads your group and ignores the `*` group entirely. A group of `Disallow: /` is harmless, because you blocked everything anyway. Write `Allow: /` under a named token and you have handed that crawler the internal paths Wix was keeping shut.

The safe shape on Wix: name a crawler only to deny it, and let the rest fall through to `*`.

## Wix's four example names, checked against 2026

Wix's article *Blocking AI Crawlers from Your Site* gives this worked example: `CCBot`, `GPTBot`, `ChatGPT-User` and `BingAI`, each with `Disallow: /`. Checked against our own [AI crawler user-agent list](https://aiscan.site/blog/ai-crawler-user-agent-list-2026):

| Token in Wix's example | What it actually does | Worth blocking? |
|---|---|---|
| `GPTBot` | OpenAI training crawler | Yes, if you are opting out of training |
| `CCBot` | Common Crawl, the open training corpus | Yes, same reason |
| `ChatGPT-User` | Live fetch triggered by a person in ChatGPT | Usually no, this is a visitor |
| `BingAI` | Appears in no operator token list we hold | No effect either way |

Missing from the example and crawling in 2026: `ClaudeBot`, `PerplexityBot`, `Google-Extended`, `meta-externalagent` and `Applebot-Extended`. Wix's own FAQ is honest about the ceiling: *"some crawlers may ignore these directives, so it's not an airtight solution."*

## Rules that keep you in the answers and out of the training set

The distinction that costs traffic is training versus retrieval. `GPTBot` trains; `OAI-SearchBot` puts you in ChatGPT's cited answers. Block the second by accident and you leave the surface you were trying to win. Paste this under **This is your current file**, below what is already there:

```
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Google-Extended
Disallow: /
```

Note the absences: `OAI-SearchBot`, `PerplexityBot` and `Bingbot` are unlisted, so they keep matching `*` and keep indexing you. One syntax rule from RFC 9309 section 2.2.1 catches people out: a product token *"MUST contain only uppercase and lowercase letters"*, plus hyphen and underscore. No slashes, no version numbers, no wildcards in the name.

## Read the file back from your own domain

Wix says to check the file by adding `/robots.txt` to your root domain. Assert the content type, not the status code:

```
curl -sI https://yoursite.com/robots.txt
```

A working file answers `200` with `content-type: text/plain`. If you get `text/html`, you are reading an error page, not a file. Then confirm your group survived the save:

```
curl -s https://yoursite.com/robots.txt | grep -A1 -i 'gptbot'
```

Expected output is the two lines you typed. An empty result means the save did not commit; reopen **View File** rather than typing it somewhere else.

## When the editor is not the problem

Wix's documentation is unusually direct that most robots.txt complaints are not robots.txt complaints. If Wix Site Inspection or Google Search Console reports **Blocked by robots.txt**, Wix says *"you don't need to edit your robots.txt file, especially if you never edited it before."*

| What you see | What it usually is |
|---|---|
| **Blocked by robots.txt** on a page you never named | Indexing switched off for that page in the editor or in SEO Settings |
| A page missing from the file's reach | The page is password protected or members only |
| Your rules are there but a bot still crawls | A crawler that ignores the protocol, which Wix concedes happens |
| A page still quoted in an AI Overview | robots.txt does not govern snippets, `nosnippet` does |

For that last row: in the editor, **Pages & Menu > More Actions > SEO basics > Advanced SEO > Robots meta tag**, and tick **nosnippet**, which Wix describes as preventing a page's content *"from being used as a direct input for AI overviews"*. One caution before denying broadly: Wix warns that blocking AI crawlers *"may limit NLWeb functionality, as AI agents need access to view your structured content."*

## Where AIScan fits, and where it doesn't

Run the scan first and read two rows: `npx aiscan-cli yoursite.com`, or paste the URL at [AIScan](https://aiscan.site/). **D1** confirms a robots.txt exists and parses at your origin root. **B2** is the row this guide is about, whether the file names AI crawlers explicitly instead of leaving them to `*`. Across our scan corpus the whole [bot-access dimension](https://aiscan.site/docs/checks/bot-access) has a median of zero, so explicit rules are still an easy differentiator.

What the scan cannot see: it fetches with its own identity, so it reads the file a browser gets rather than the file a given crawler gets, and it cannot tell you whether a crawler obeyed. That needs server logs, which Wix does not expose.

Wix hosts at the origin root, which spares you the failure mode that makes this hard elsewhere: on a docs framework deployed under a repository path the same file becomes void, as covered in [robots.txt rules for AI bots on Docusaurus](https://aiscan.site/blog/robots-txt-ai-bots-docusaurus). If you also run a WordPress blog next to the Wix site, [ThinkRank](https://thinkrank.ai) handles robots.txt, robots meta, schema and llms.txt from one plugin there and migrates settings from Rank Math, Yoast, AIOSEO and SEOPress; Rank Math and Yoast both edit robots.txt too.

## Publish, then scan

Save the file, publish the site, and re-run `npx aiscan-cli yoursite.com`. Read **B2** first and **D1** second. If B2 still fails, fetch the file and check your named groups are spelled with the exact tokens above, because a typo in a product token is ignored silently rather than reported. More platform walkthroughs are in the [AIScan guides](https://aiscan.site/guides).