How to declare allow/deny rules for named AI crawlers in robots.txt on Wix

Wix writes your robots.txt for you. Fetched on 7 September 2026, the file at www.wix.com/robots.txt ran to 90 non-empty lines: one User-agent: * group, 87 Disallow rules, and a single Sitemap: line at the bottom. Adding a rule for GPTBot or ClaudeBot takes about two minutes in the Robots.txt Editor, and it does something most guides never mention: the crawler you just named stops reading every one of those 87 lines. That trade is the whole of this job on Wix.

Quick summary

Wix names four crawlers in its own worked example. One of the four appears in no operator token list we hold.

Wix keeps rewriting the file you are about to edit

According to Wix's article Editing Your Site's Robots.txt File, read on 7 September 2026, the platform maintains the file continuously: "If you need to update a page, Wix automatically updates your robots.txt after you publish the page. If you change your site's settings, your robots.txt file is updated immediately."

That is the opposite of how the other generated file on Wix behaves. As covered in publishing a valid llms.txt on Wix, Wix says that once you edit llms.txt, "it will stop updating automatically so your changes are preserved." Robots.txt carries no such lock: your named groups sit alongside a * group Wix goes on regenerating around them. Wix also puts a warning above the procedure worth repeating: "Wix Customer Care cannot help you with any changes you make to your robots.txt file."

Open the Robots.txt Editor

Wix documents one path for reading, editing and resetting:

1. Go to SEO & GEO in your site's dashboard.
2. Scroll down to Tools and settings.
3. Click Robots.txt Editor.
4. Click View File.
5. Type your directives under This is your current file.
6. Click Save Changes, then Save.

To undo everything, the same four clicks then Reset to Default and Reset. There is no file manager and no FTP on Wix, so this panel is the only way in.

A named group replaces the defaults, it does not extend them

RFC 9309 section 2.2.1 states verbatim: "Crawlers MUST use case-insensitive matching to find the group that matches the product token and then obey the rules of the group." Groups for different tokens do not stack; the spec merges only groups matching the same token.

Every rule Wix generated sits under User-agent: *. The moment you add User-agent: GPTBot, that crawler reads your group and ignores the * group entirely. A group of Disallow: / is harmless, because you blocked everything anyway. Write Allow: / under a named token and you have handed that crawler the internal paths Wix was keeping shut.

The safe shape on Wix: name a crawler only to deny it, and let the rest fall through to *.

Wix's four example names, checked against 2026

Wix's article Blocking AI Crawlers from Your Site gives this worked example: CCBot, GPTBot, ChatGPT-User and BingAI, each with Disallow: /. Checked against our own AI crawler user-agent list:

Missing from the example and crawling in 2026: ClaudeBot, PerplexityBot, Google-Extended, meta-externalagent and Applebot-Extended. Wix's own FAQ is honest about the ceiling: "some crawlers may ignore these directives, so it's not an airtight solution."

Rules that keep you in the answers and out of the training set

The distinction that costs traffic is training versus retrieval. GPTBot trains; OAI-SearchBot puts you in ChatGPT's cited answers. Block the second by accident and you leave the surface you were trying to win. Paste this under This is your current file, below what is already there:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Google-Extended
Disallow: /

Note the absences: OAI-SearchBot, PerplexityBot and Bingbot are unlisted, so they keep matching * and keep indexing you. One syntax rule from RFC 9309 section 2.2.1 catches people out: a product token "MUST contain only uppercase and lowercase letters", plus hyphen and underscore. No slashes, no version numbers, no wildcards in the name.

Read the file back from your own domain

Wix says to check the file by adding /robots.txt to your root domain. Assert the content type, not the status code:

curl -sI https://yoursite.com/robots.txt

A working file answers 200 with content-type: text/plain. If you get text/html, you are reading an error page, not a file. Then confirm your group survived the save:

curl -s https://yoursite.com/robots.txt | grep -A1 -i 'gptbot'

Expected output is the two lines you typed. An empty result means the save did not commit; reopen View File rather than typing it somewhere else.

When the editor is not the problem

Wix's documentation is unusually direct that most robots.txt complaints are not robots.txt complaints. If Wix Site Inspection or Google Search Console reports Blocked by robots.txt, Wix says "you don't need to edit your robots.txt file, especially if you never edited it before."

For that last row: in the editor, Pages & Menu > More Actions > SEO basics > Advanced SEO > Robots meta tag, and tick nosnippet, which Wix describes as preventing a page's content "from being used as a direct input for AI overviews". One caution before denying broadly: Wix warns that blocking AI crawlers "may limit NLWeb functionality, as AI agents need access to view your structured content."

Where AIScan fits, and where it doesn't

Run the scan first and read two rows: npx aiscan-cli yoursite.com, or paste the URL at AIScan. D1 confirms a robots.txt exists and parses at your origin root. B2 is the row this guide is about, whether the file names AI crawlers explicitly instead of leaving them to *. Across our scan corpus the whole bot-access dimension has a median of zero, so explicit rules are still an easy differentiator.

What the scan cannot see: it fetches with its own identity, so it reads the file a browser gets rather than the file a given crawler gets, and it cannot tell you whether a crawler obeyed. That needs server logs, which Wix does not expose.

Wix hosts at the origin root, which spares you the failure mode that makes this hard elsewhere: on a docs framework deployed under a repository path the same file becomes void, as covered in robots.txt rules for AI bots on Docusaurus. If you also run a WordPress blog next to the Wix site, ThinkRank handles robots.txt, robots meta, schema and llms.txt from one plugin there and migrates settings from Rank Math, Yoast, AIOSEO and SEOPress; Rank Math and Yoast both edit robots.txt too.

Publish, then scan

Save the file, publish the site, and re-run npx aiscan-cli yoursite.com. Read B2 first and D1 second. If B2 still fails, fetch the file and check your named groups are spelled with the exact tokens above, because a typo in a product token is ignored silently rather than reported. More platform walkthroughs are in the AIScan guides.