---
title: "Cloudflare Pay Per Crawl in 2026: Should You Charge AI Crawlers? 119 Sites Measured"
slug: cloudflare-pay-per-crawl-should-you-charge
published: 2026-09-06T12:04:15.904128+00:00
updated: 2026-09-06T12:04:15.904128+00:00
author: "Asif Rahman"
author_url: https://masifrahman.com
category: "AI Readiness"
tags: check:M3, check:M4, check:M1, check:B1, check:B2, check:B3, platform:wordpress, platform:shopify, Cloudflare, pay per crawl, x402, HTTP 402, AI crawlers, Web Bot Auth, content licensing
description: "Stack Overflow charges AI crawlers $0.50 a page. We probed 119 publishers: 16 returned HTTP 402, only 6 named a price. Here is what Pay Per Crawl costs."
url: https://aiscan.site/blog/cloudflare-pay-per-crawl-should-you-charge
---

Stack Overflow will sell you its homepage for fifty cents. Point an AI crawler at it and the server answers `HTTP 402 Payment Required`, `crawler-price: USD 0.5`, and a JSON body naming Cloudflare's Pay Per Crawl by name. The status code that RFC 9110 still describes as "reserved for future use" is now a live price tag on one of the twenty most-visited sites on the web.

That changes the question this article was going to ask. "Should you charge AI crawlers" stopped being hypothetical some time before 6 September 2026, and the useful version now is narrower: what does charging actually get you, what does it cost you, and what should you do first instead.

To answer it we probed 119 publisher and reference sites with an AI crawler user agent on 6 September 2026 and recorded exactly what came back. Sixteen asked for money. Only six named a price.

## Quick summary

| If you want to… | Do this | What it costs | What it gets you |
|---|---|---|---|
| Find out whether you are already being crawled | Read AI Crawl Control's Crawlers tab, free on every Cloudflare plan | Nothing | Per-bot request counts and robots.txt violations |
| Cut the cost of crawls you already serve | Ship `ETag` and `Last-Modified`, answer `304` | An afternoon | 77.5% fewer bytes on a re-crawl, measured across 195 responses |
| Say "this content is licensable" without billing | Set the AI Crawl Control block response to `402` with a custom body | Paid Cloudflare plan | A licensing inbox, no revenue |
| Actually collect money per crawl | Cloudflare Pay Per Crawl | Closed beta, minimum $0.001 per crawl | Payment from crawlers that hold a Cloudflare billing relationship and sign with Web Bot Auth |
| Sell one machine-readable thing rather than pages | An x402 endpoint on your own API | Wallet plus facilitator | A price agents can pay with no account |

**The honest answer for most readers is still "not yet".** Pay Per Crawl remains a closed beta, [according to Cloudflare's own documentation](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) as fetched on 6 September 2026, and it only bills crawlers that have already onboarded to Cloudflare and identified themselves with Web Bot Auth. The people it works for today are the ones with content valuable enough that Anthropic, Perplexity and Meta would rather pay than skip.

**Best for large publishers:** Pay Per Crawl, because you have the traffic and the bargaining power.
**Best for everyone else:** a `402` block response with a licensing address in it, which costs nothing to run and opens a conversation.
**Best first move for every site:** conditional requests, because they reduce the bill you are already paying.

## What a 402 actually looks like on the wire

Here is the whole thing, fetched from `stackoverflow.com` on 6 September 2026 by a request carrying a ClaudeBot user agent:

```
HTTP/2 402
date: Sun, 06 Sep 2026 03:10:56 GMT
content-type: application/json
crawler-error: MissingCrawlerPrice
crawler-price: USD 0.5
server: cloudflare
```

And the body:

```json
{"x402Version":0,
 "accepts":[{"scheme":"deferred","network":"cloudflare","resource":"/",
             "maxTimeoutSeconds":30,"maxAmountRequired":"0.5","asset":"USD"}],
 "error":"Payment is required to access content. Please refer to the Cloudflare Pay Per Crawl documentation for more information."}
```

Two things in there are worth stopping on.

The `crawler-price` and `crawler-error` headers are the ones named, verbatim, on Cloudflare's [Crawl pages](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/use-pay-per-crawl-as-ai-owner/crawl-pages/) reference, down to the error code. This is not a lookalike. It is the product.

The body is an **x402 payload**. Cloudflare's Pay Per Crawl documentation never uses the word x402 anywhere, but the wire format announces itself as `x402Version: 0` with `scheme: "deferred"` and `network: "cloudflare"`. The protocol everyone associates with crypto micropayments is also the envelope Cloudflare uses for a Stripe-settled invoice. Nobody appears to have published that before, and it matters for anyone building a crawler: the same parser handles both.

[RFC 9110 §15.5.3](https://www.rfc-editor.org/rfc/rfc9110.txt), the current HTTP semantics standard, still says of this status code: *"The 402 (Payment Required) status code is reserved for future use."* The future turned up without an errata.

## We probed 119 publishers as an AI crawler. Sixteen asked for money.

**Method.** 119 news, magazine, reference, recipe, finance and Q&A sites, chosen by hand from public brands and kept entirely separate from our scan corpus. One `GET /` each, user agent `ClaudeBot/1.0`, no Web Bot Auth signature, 6 September 2026. We recorded the status code, the `crawler-price` and `crawler-error` headers, the `server` header and the response body. A second pass sent fourteen different crawler user agents at the two most interesting hosts.

| Response to an AI crawler | Sites | Share |
|---|---|---|
| `200` served normally | 60 | 50.4% |
| `403` refused | 24 | 20.2% |
| **`402` Payment Required** | **16** | **13.4%** |
| `301` / `302` / `307` redirect | 16 | 13.4% |
| `401` or other | 3 | 2.5% |

Sixteen of 119 is higher than we expected and much higher than any published figure. But the split inside those sixteen is the real finding.

| 402 group | Sites | Price attached? | What the body says |
|---|---|---|---|
| Stack Exchange network | 6 | **Yes, `USD 0.5`** | x402 payload naming Cloudflare Pay Per Crawl |
| People Inc titles | 6 | No | "please contact contentlicensing@people.inc" |
| National newspapers | 3 | No | Access restricted, security notice, or a licensing partner's message |
| Other | 1 | No | Generic refusal |

**Only 6 of 119 sites, 5.0%, quoted a machine-readable price.** All six are Stack Exchange properties: `stackoverflow.com`, `stackexchange.com`, `superuser.com`, `serverfault.com`, `askubuntu.com` and `math.stackexchange.com`, all at `USD 0.5`, all with identical headers. One publisher, one decision, six domains.

Three caveats travel with these numbers. The sample is a hand-built list of large English-language publishers and is not representative of the web. An unsigned request is not what a real crawler sends, so a site could behave differently for a verified bot. And a `402` seen from one datacentre IP on one day is a snapshot, not a policy.

![The four-step Pay Per Crawl negotiation captured live from stackoverflow.com on 6 September 2026, alongside the status codes 119 publishers returned to an AI crawler and AIScan's M3 pass rate across 488 scanned sites](https://gqdxuwsuuasfjeeqafyq.supabase.co/storage/v1/object/public/blog-covers/cloudflare-pay-per-crawl-should-you-charge-fig1.jpg)

## Two ways to say "pay me", and only one of them has a price

The ten priceless 402s are not failures. They are a different product being used deliberately.

Cloudflare's [Manage AI crawlers](https://developers.cloudflare.com/ai-crawl-control/features/manage-ai-crawlers/) page, last updated 28 July 2026, lets any paid-plan customer choose the status code returned to a blocked crawler from exactly two values, verbatim: *"**403 Forbidden**: Use this option if you wish to indicate that you do not want the AI crawler to access your content. **402 Payment Required**: Use this option if you wish to indicate that the AI crawler must pay to access your content."* You can also write the response body, and Cloudflare says why: *"This provides you with a channel to open dialogue with the AI crawler owner, and to inform the AI crawler how to properly license their content, thereby creating a direct path from crawling attempt to commercial agreement."*

Six sites in our sample are running that feature word for word. `allrecipes.com`, `investopedia.com`, `realsimple.com`, `seriouseats.com`, `thespruce.com` and `verywellhealth.com` all return `402` to an AI crawler with a byte-identical body:

```
Payment Required - If you wish to license content from People Inc,
please contact contentlicensing@people.inc
```

That is a licensing desk implemented as an HTTP status code, and it needs no beta invitation. `www.theatlantic.com` runs a plainer version, `{"message":"Please contact the site owner for access."}`. `www.usatoday.com` returns `402` with a message pointing at a third-party toll service rather than at Cloudflare, which is worth knowing before anyone reads a `402` as proof of Pay Per Crawl. And `www.telegraph.co.uk` returns `402` attached to an unusual-activity security notice, which reads more like a misconfigured challenge than a commercial position.

So the practical taxonomy for a `402` in 2026 is three ways, not one: a real per-crawl price, a licensing invitation, or a bot challenge wearing the wrong number.

## Who Stack Overflow charges, and who it walks straight in

Cloudflare lets a site owner pick an action per crawler. We sent fourteen crawler user agents at `stackoverflow.com` and at `www.theatlantic.com` on 6 September 2026 to see how that looks from outside.

| Crawler | stackoverflow.com | www.theatlantic.com |
|---|---|---|
| GPTBot | `302` allowed | `200` allowed |
| OAI-SearchBot | `302` allowed | `200` allowed |
| ChatGPT-User | `302` allowed | `200` allowed |
| Googlebot | `302` allowed | `200` allowed |
| Bingbot | `302` allowed | `200` allowed |
| Amazonbot | `302` allowed | `200` allowed |
| Applebot | `302` allowed | **`402`** |
| ClaudeBot | **`402` USD 0.5** | **`402`** |
| Claude-SearchBot | **`402` USD 0.5** | **`402`** |
| Claude-User | **`402` USD 0.5** | **`402`** |
| PerplexityBot | **`402` USD 0.5** | **`402`** |
| meta-externalagent | **`402` USD 0.5** | **`402`** |
| Bytespider | **`402` USD 0.5** | **`402`** |
| CCBot | `403` blocked | **`402`** |

Both publishers let all three OpenAI tokens through free and charge all three Anthropic tokens. Both let the two classic search engines through untouched, which matches Cloudflare's own warning on the [Select crawlers to charge](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/use-pay-per-crawl-as-site-owner/select-crawlers-to-charge/) page that setting a Search Engine Crawler to Block or Charge *"may negatively impact your site's SEO performance"*.

Two things this table does not prove. It does not prove any commercial agreement exists behind the OpenAI allowance, only that the allowance exists. And a user agent string is a claim, not an identity, which is the whole reason Pay Per Crawl bills on Web Bot Auth signatures rather than on the `User-Agent` header. If you want to know which tokens exist and how far you can trust one, our [AI crawler user agent list](https://aiscan.site/blog/ai-crawler-user-agent-list-2026) has the verified inventory and the reverse-DNS tests that show where verification breaks.

One correction we owe the reader, because we nearly published it wrong. `stackoverflow.com/questions` returns `403` to an AI crawler, which looks like a targeted block on the answers. It is not. The response carries `cf-mitigated: challenge` and a "Just a moment…" interstitial, and an ordinary desktop browser user agent from the same address gets the same `403`. That is a managed challenge on a path, not a pricing decision. The short answer URLs `/q/<id>` and `/a/<id>` do return `402` at `USD 0.5`.

## What the docs say it costs, and what the dashboard does not show you

All of this is fetched from Cloudflare's own documentation on 6 September 2026, and quoted in its own words where the wording matters.

| Mechanic | What Cloudflare documents |
|---|---|
| Availability | Closed beta. Sign up at the Pay Per Crawl page or through an Enterprise account executive |
| Minimum price | **$0.001 USD per crawl** |
| Price granularity | One price per zone. The FAQ is explicit: *"you can only set a single price that applies to all crawlers configured with the 'Charge' option"* |
| Charging event | A successful `HTTP 200` content retrieval. *"Error responses are not billed"* |
| Re-crawls | Billed again. *"Every time your AI crawler accesses content on a website protected with pay per crawl, it will incur the cost set by the site owner"* |
| Always-free paths | `/robots.txt`, `/sitemap.xml`, `/security.txt`, `/.well-known/security.txt`, `/crawlers.json` |
| Per-path pricing | Optional. Enable dynamic pricing, then return a `crawler-price` response header; Cloudflare sends `cf-pay-per-crawl: protocol=cloudflare, pricing=in-band` on every origin request so your code knows when to set it |
| Free sections | Configuration Rules with a **Disable Pay Per Crawl** action on a URI pattern |
| Money out | A dedicated Cloudflare Stripe Connect account. *"Pre-existing Stripe accounts are not compatible with this feature."* Monthly payouts to publishers in good standing |
| Precedence | A WAF or Bot Management block overrides Charge. The blocked crawler gets nothing, and you get nothing |

And the sentence a publisher should read twice, from the [Manage payouts](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/use-pay-per-crawl-as-site-owner/manage-payouts/) page: *"Your accrued balance is not currently visible in the dashboard. You can request balance updates from your Cloudflare team."* Payouts are also *"subject to settlement periods and minimum payout thresholds"* that the documentation does not quantify.

That is a real limitation and it deserves to be said plainly. You can set a price today and not be able to see what you have earned without emailing someone.

The buyer side is gated just as hard. According to the same documentation set, Cloudflare publishes a Discovery API at `crawlers-api.ai-audit.cfdata.org/charged_zones` that lets a crawler list every domain currently charging. We called it on 6 September 2026 and got `HTTP 403` with `{"error":"Only verified bots can use this endpoint"}`. So there is a directory of who is charging, and publishers cannot read it. Neither can we, which is why the sweep above had to be done the slow way.

## The error codes have drifted from the documentation

Because Stack Overflow is live, the whole payment negotiation can be walked through with `curl`. We did, unsigned, so every attempt fails at the authentication step, which is exactly what makes the error codes readable.

| What we sent | What came back | Cloudflare's documented behaviour |
|---|---|---|
| Nothing | `402` `MissingCrawlerPrice`, `crawler-price: USD 0.5` | Matches: 402, "include either header" |
| `crawler-max-price: USD 1.00` | `403` `PaymentFailed` | Matches: 403 |
| `crawler-exact-price: USD 0.5` | `403` `PaymentFailed` | Matches: 403 |
| `crawler-max-price: USD 0.10` | `402` **`InsufficientCrawlerMaxPrice`** | **Undocumented code.** The published table lists `InvalidCrawlerMaxPrice` for this case |
| Both price headers at once | `400` `ConflictingPriceHeaders` | Matches: 400 |
| `crawler-max-price: 1.00` (no currency) | `400` `InvalidCrawlerMaxPrice` | **Drifted.** The table gives this code `402` and a different meaning; `InvalidCrawlerPriceValue` is the documented 400 |

Two of six responses do not match the published [error code reference](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/use-pay-per-crawl-as-ai-owner/error-codes/), which was last updated 23 April 2026 while the pages around it moved on 28 July. If you are writing a crawler that pays, do not switch on `crawler-error` string equality against that table alone.

The `403 PaymentFailed` result is the important one for publishers. An unsigned request offering the full asking price still gets nothing. Pay Per Crawl bills identity, not intent, and the identity is a Web Bot Auth signature over signed components that must include the payment header. That is why you cannot audit anyone's crawl prices at scale, and why a search for "how many sites charge for crawls" returns nothing useful.

## The file you publish for agents is billable. robots.txt is not.

Look again at Cloudflare's always-free list: `/robots.txt`, `/sitemap.xml`, `/security.txt`, `/.well-known/security.txt`, `/crawlers.json`.

`llms.txt` is not on it. Neither is `agents.md`, nor `/.well-known/mcp/server-card.json`, nor any of the other discovery files the agentic web has invented since 2024.

We checked on the one site where it can be checked. `stackoverflow.com/llms.txt` returns `402` with `crawler-price: USD 0.5`, the same price as an article. A site that turns on Pay Per Crawl charges half a dollar for the map it wrote specifically to help agents find their way around, while handing over `robots.txt` for free.

For most publishers this is a small thing that will bite once. The fix is a Configuration Rule with **Disable Pay Per Crawl** on your discovery paths, and it takes a minute. It is also a gap Cloudflare could close by adding those paths to the free list, and until it does, anyone comparing discovery files across the web will get a distorted picture. If you are still deciding which of these files to publish at all, our breakdown of [llms.txt against robots.txt and sitemap.xml](https://aiscan.site/blog/llms-txt-vs-robots-txt-vs-sitemap) covers what each one is actually for, and the [llms.txt generator](https://aiscan.site/llms-txt-generator) will write you a spec-shaped file.

One more oddity from the same host, offered with a caveat. `stackoverflow.com/robots.txt` answered our requests with `HTTP 418` and a 113-byte body reading `User-agent: *`, `Content-signal: search=no, ai-train=no`, `Disallow: /`. The same 418 came back for `stackexchange.com` and `askubuntu.com`, byte-identical. We cannot tell from one network whether that status is a deliberate configuration or IP reputation, so treat it as an observation rather than a finding. What is not in doubt is the effect if it is general: Google's robots.txt handling treats most 4xx responses as "no restrictions", so a file saying `Disallow: /` delivered with a 418 tells a compliant crawler nothing at all.

## Cloudflare's ledger and Coinbase's chain are both called x402

The queue note that started this article assumed Cloudflare Pay Per Crawl and x402 were two competing answers. They are one protocol with two settlement layers, and putting the payloads side by side is the clearest way to price the difference.

| | Cloudflare Pay Per Crawl | Agent Ready's x402 endpoint |
|---|---|---|
| Where we read it | `402` body from `stackoverflow.com`, 6 Sep 2026 | `agent-ready.dev/.well-known/x402`, 6 Sep 2026 |
| Protocol version | `x402Version: 0` | `x402Version: 2` |
| Scheme | `deferred` | `exact` |
| Network | `cloudflare` | `eip155:8453` (Base) |
| Asset | `USD` | USDC at `0x8335…2913` |
| Price | `0.5` per page | `20000` and `250000` base units, so $0.02 and $0.25 |
| Unit sold | One page fetch | One 25-page or 250-page scan |
| Who can buy | Verified Web Bot Auth crawlers with a Cloudflare account | Any agent with a wallet |
| Who gets paid | Publisher, monthly, via Stripe Connect | Merchant, on settlement, via a Coinbase facilitator |
| Account needed | Yes, on both sides | No |

Agent Ready's own wording, [verified on its pricing page](https://agent-ready.dev/pricing) on 6 September 2026: *"$0.02 USDC for a 25-page scan or $0.25 for a 250-page scan, settled on Base mainnet (the facilitator pays the gas)."*

The comparison that follows is uncomfortable for the per-crawl model. Agent Ready sells a *result* for two cents. Stack Overflow sells a *page* for fifty. A crawler that needs twenty-five pages of context pays $0.02 in one case and $12.50 in the other. Per-crawl pricing is only rational where the page itself is the scarce thing, which is true for a newspaper archive and false for almost everything else.

For scale on the wider protocol, [x402.org](https://www.x402.org/) reported **75.41M transactions and $24.24M of volume across 94.06K buyers and 22K sellers in the trailing 30 days** as displayed on 6 September 2026. Read that as x402 payments of every kind, not as crawl payments. We found no public figure for how many crawls have ever been paid for, and Cloudflare has not published one.

## Before you charge, stop paying for the crawls you do not need

There is a free lever almost nobody pulls, and a post about charging that skips it is a sales pitch.

A crawler that re-fetches an unchanged page costs you the full payload every time. The fix is thirty years old: send `ETag` and `Last-Modified`, honour `If-None-Match` and `If-Modified-Since`, and answer `304`. According to Google Search Central's own crawling guidance, `ETag` is the recommended validator by name.

We measured how many sites can do it. Across 195 successful responses from 60 hosts on 5 September 2026, **68.2% shipped a cache validator, only 51.8% actually answered `304`, and a conditional re-crawl moved 77.5% fewer bytes** than a cold one. The full method and the per-surface breakdown are in [AI crawler traffic in 2026](https://aiscan.site/blog/ai-crawler-traffic-waste-2026).

The interaction with Pay Per Crawl is genuinely interesting and genuinely undocumented. Cloudflare, in its own FAQ, defines the charging event as a successful `HTTP 200` content retrieval and says verbatim that *"error responses are not billed"*. A `304 Not Modified` is neither. The documentation does not say which side of the line it falls on, so a publisher charging per crawl cannot currently tell whether a well-behaved crawler that revalidates is a customer or a freeloader. That is a question worth asking Cloudflare before you price anything.

Either way, the sequence is the same and it is not the one most posts recommend:

1. **Make each fetch cheap.** Cache validators, correct `304`s. Free, standards-based, works with every crawler including the ones that ignore you.
2. **Say what you want.** Explicit robots.txt rules for named AI agents, plus Content Signals. Our corpus says only 37.3% of sites do this at all.
3. **Rate-limit the ones that misbehave.** AI Crawl Control's Crawlers tab shows you who they are, free on every plan.
4. **Block what you will never license.** `403`, or `402` with a licensing address if you would rather talk.
5. **Charge.** Only once steps one to four are done and the traffic still justifies it.

On WordPress, [ThinkRank](https://thinkrank.ai) is the shortest route through steps two and three. One plugin owns robots.txt, robots meta, schema, sitemaps and the llms.txt file together, which matters here because the usual WordPress mess is three SEO plugins fighting over one robots.txt while you try to add a single AI directive to it. Existing configuration imports across from Rank Math, Yoast, All in One SEO or SEOPress, so nothing has to be typed twice. Yoast and Rank Math both carry larger template libraries and longer histories on classic SEO reporting; if that is your daily work they stay reasonable picks, and neither of them writes the llms.txt. On Shopify, [StoreSEO](https://storeseo.com/) covers llms.txt, agents.md, schema and the AEO side that Shopify's own agent rollout does not touch. It is rated 5.0 from 737 reviews with the Built for Shopify badge on [the Shopify App Store](https://apps.shopify.com/storeseo) as of 6 September 2026.

## Should you charge? A threshold, not a slogan

| Your situation | Charge? | What to do instead or as well |
|---|---|---|
| Large publisher, archive with licensing value, already fielding inbound licensing mail | **Yes, apply for the beta** | Exclude discovery paths with a Configuration Rule first |
| Publisher with valuable content but no beta access | Not yet | `402` block response with a licensing address, exactly as People Inc runs it |
| SaaS or product site living on organic discovery | **No** | Charging the crawlers that cite you costs you citations |
| Docs site | **No** | Your content is your funnel. Make it cheap to fetch and easy to parse |
| Store | **No** | Sell products to agents, not pages. UCP and MCP endpoints, not a toll |
| Anyone whose bandwidth bill is the actual complaint | **No** | Conditional requests, then rate limits |

The test is not whether AI companies should pay. It is whether *your* pages are worth more as revenue than as citations. For Stack Overflow, whose content is the answer a developer wanted, fifty cents is a defensible number. For a company blog whose job is to be quoted, a `402` is a hole in the funnel.

Two more constraints before anyone applies. A single zone price means you cannot charge a training crawler more than a search crawler, which is the distinction most publishers actually care about. And because the WAF overrides Charge, an existing bot rule can silently turn your revenue into a `403`.

## Where AIScan fits, and where it doesn't

Start with the scan. Run `npx aiscan-cli yoursite.com` from a terminal, or drop your address into the box on [aiscan.site](https://aiscan.site/). No signup and no card. It grades the parts of this that a machine can check:

- **M3, x402 payment signal** and **M4, machine-readable pricing** on [/docs/checks/commerce](https://aiscan.site/docs/checks/commerce)
- **B1 Content Signals**, **B2 explicit AI bot rules** and **B3 Web Bot Auth key directory** on [/docs/checks/bot-access](https://aiscan.site/docs/checks/bot-access)

Now the part we would rather not print. Across **488 sites scanned on rubric `2026.08.2` between 25 August and 6 September 2026**, one latest scan per host with our own pages excluded, **M3 passes on 0 of the 180 sites our classifier treats as commerce.** Not one, on the third consecutive week of asking. M4 passes on 8.9% and M1 on 15.0%. On the bot access side, B2 passes on 37.3% of all 488 sites, B1 on 21.5%, and B3 on 3.9%.

**AIScan does not grade a `402` at all.** None of our 21 checks looks for a `crawler-price` header, a Pay Per Crawl block response, or a licensing message in a refusal body. The sixteen sites in the sweep above that asked an AI crawler for money would get no credit from us for it, and the six with a real price would score identically to the ten without one. That is a genuine gap, it is now on our backlog, and until it ships you cannot use our score to answer this article's question.

We also do not charge. `aiscan.site` returns `200` to a ClaudeBot request, and as we published last week it ships no cache validator either, so we fail our own step one before we get anywhere near step five.

Cloudflare's free scanner at `isitagentready.com` does grade `x402` as one of five commerce protocol checks, but only on sites its classifier decides are commerce, and it misfires. Use it as a second opinion, not as an audit.

## What to do this week

Run the scan and read the commerce and bot access rows. Then, in this order: check whether your origin returns `304` to a conditional request, and fix it if not; look at AI Crawl Control's Crawlers tab to see who is actually fetching you; and only then decide whether any of them should be paying. If the answer is yes, exclude your discovery files from charging before you enable anything.

The single command that tells you where you stand:

```bash
npx aiscan-cli yoursite.com
```

More setup walkthroughs are at [aiscan.site/guides](https://aiscan.site/guides), including the [WordPress](https://aiscan.site/docs/platforms/wordpress) and [Shopify](https://aiscan.site/docs/platforms/shopify) paths. If you want the wider argument about what crawlers cost you before any money changes hands, [The State of AI Agent Readiness](https://aiscan.site/blog/state-of-ai-agent-readiness-2026) has the distribution, and [Cloudflare's three crawler categories](https://aiscan.site/blog/cloudflare-search-agent-training-crawler-categories) explains the Search, Agent and Training split that decides which bots a charge would even apply to.