The Agent-Readiness Fix That Moves Your Score Most (2026): 681 Sites Ranked

Every agent-readiness checklist ends the same way: twenty-odd items, all marked "important", no order. A site owner with one free afternoon cannot do twenty things, and nobody tells them which one to do first. We can answer that with data, because AIScan has graded the same rubric across hundreds of real sites. On 4 October 2026 we queried our own production corpus: 681 distinct hosts, one latest scan per host, all on rubric 2026.08.2, with our own domain excluded. The mean score is 55.5 out of 100. This post ranks the fixes by what they are worth in that data, and then admits where the ranking is weak.

The short version of the answer is not the one the raw numbers suggest. Sites that pass a given check score roughly 30 to 40 points higher than sites that fail it, for almost every check. That gap is mostly a sign of who passes, not of what the check is worth. Once we compare sites of similar overall maturity, four fixes stand out for every site, three more stand out for sites with an API, and the rest shrink to noise.

Quick summary

What the data says: C1, B2 and C2 alone account for 8,150 of the 16,113 weight points that failing checks cost across the whole corpus, 50.6%. Add D3 and the figure is 56.3%. Those four are failed often and still separate similar sites from each other. Nothing else in the rubric manages both.

Best for a WordPress owner: B2 and C2 in one sitting, using ThinkRank, which manages robots.txt and llms.txt in one plugin. Best for a Shopify merchant: C2 and B2 through StoreSEO, then leave C1 alone until your platform exposes it. Best for a developer with a server: C1, because it is the single most expensive failure in the rubric.

How this ranking was built

Method first, because the limits decide how far you can trust the rest. All figures were verified on 4 October 2026 against the live production database.

The corpus is every scan AIScan has stored under rubric version 2026.08.2, reduced to the most recent scan per host. Hosts are normalised by stripping the scheme and a leading www.. We excluded aiscan.site, which has scanned itself 34 times at 100 and would move the mean by about five points. That leaves 681 hosts. Of those, 449 carry the 18-check version of the rubric and 232 carry 21 checks because commerce applies. We never divide a commerce check by the full 681; M1, M3 and M4 use 231.

Everything below came from aggregate SQL. No row-level query was run and no individual host is named anywhere in this post. The shape of the main query is a common table expression that unnests the report's check array and groups by check ID:

WITH latest AS (
  SELECT DISTINCT ON (host) host, score, result
  FROM scans WHERE rubric_version = '2026.08.2' AND host <> 'aiscan.site'
  ORDER BY host, created_at DESC
)
SELECT c->>'id' AS id,
       count(*) FILTER (WHERE c->>'status' = 'pass') AS pass,
       avg(score) FILTER (WHERE c->>'status' = 'pass') AS mean_pass,
       avg(score) FILTER (WHERE c->>'status' <> 'pass') AS mean_not_pass
FROM latest, jsonb_array_elements(result->'checks') c
GROUP BY 1;

Each check in the report carries a status (pass, partial, fail or info), a weight and an earned value. info means the check reports something but is not scored: on those rows the weight is 0 and nothing is lost. The bonus checks (B1, B3, P2, P3, M1, M3) are info on most hosts, so for them a missing file costs nothing and only a pass earns anything. That distinction matters later. It lets us measure the weight a host actually leaves on the table. What we cannot do is convert that weight into overall score points one for one. We tested it: the share of weight a host earns correlates with its final score at 0.95 on 18-check scans and 0.89 on 21-check scans, but the mean absolute difference is 11.8 and 16.4 points. The scorer applies tiers and level rules on top of the weights, so we report weight in "weight points" and the score gap separately.

This is an observational study. Sites were scanned because their owners, or we, chose to scan them, so the corpus skews towards people who already care about the topic. Read the ranking as a triage order. It is not a causal claim that adding a file raises your score by exactly the gap shown.

What the corpus looks like

According to the scorer's own level bands, every score maps to one of six levels. Here is where the 681 hosts sit, according to the stored level field on each latest scan:

The middle is crowded. 358 hosts, 52.6% of the corpus, sit at Level 2 or 3, which is where one well-chosen fix can change the label. Hosts without commerce checks average 50.3; hosts with them average 65.6. That 15-point difference is mostly composition, since the stores in the corpus are a self-selected group who are further along than the typical blog. The State of AI Agent Readiness in 2026 covers the pass rates per check in detail, and earlier posts used different cuts of the corpus, so a figure here can differ slightly from theirs.

Step one: where the weight is lost

Start with plain accounting. For every scored check, the table counts the hosts in each state and sums the weight lost on fail and partial rows. Checks are sorted by total weight lost across the 681 hosts.

Commerce checks apply to 231 hosts. M4, machine-readable pricing, lost 865 weight points there (29 pass, 100 partial, 102 fail). M1 is info on 197 of 231 and M3, the x402 payment signal, is info on all 231, so neither costs anything and M3 separates no one. P2 and B3 lose little (165 and 102 points) because 604 and 610 hosts respectively report info.

According to the weights stored on each scan, C1 stands apart. On every failing scan it is worth 8 weight points, more than any other check in the table, and 478 hosts do not pass it: 472 fail and 6 earn partial credit. It alone accounts for 23.6% of all weight lost. B2 and C2 follow, and the three together are half the loss.

Notice what the table does not say. A check near the top of this list is failed often, and that is all it says. D1 loses 802 points because 191 hosts have a missing or broken robots.txt. It does not say fixing D1 separates one site from another.

Step two: the raw gap lies

The tempting next step is to compare the mean score of hosts that pass a check with the mean of hosts that fail or earn partial credit. Here is that comparison for the checks that matter most. It is nearly useless:

Almost every row lands between 26 and 43 points. D2, the XML sitemap, shows one of the biggest raw gaps, and a sitemap is a file nearly every competent site already has. The gap is real, but it is not the sitemap's doing. It exists because a site that publishes a sitemap also tends to have a clean robots.txt, a canonical host and sane headings. Passing one check is a marker of passing many.

The data shows it directly. Hosts that pass D2 pass 8.4 other checks on average; hosts that fail it pass 2.1. Hosts that pass C1 pass 10.1 others, hosts that fail it 5.3. The pass-versus-fail comparison is mostly a comparison of mature sites against neglected ones.

To strip that out, we held maturity roughly constant. We took only hosts that pass between 6 and 10 checks in total, a band of 320 hosts, and repeated the comparison inside it. Hosts in that band look alike: they have the basics and are missing the newer items. Now the checks separate:

Raw gaps of 26 to 43 collapse. D1, D2, C3 and E3 drop to a few points or less, and E1, E4 and E5 turn slightly negative. Inside this band almost everyone already passes the hygiene checks, so passing them does not distinguish you. They are table stakes: failing them hurts, passing them is expected.

The checks that survive split into two groups. C2, C1, B2 and D3 apply to every site, and each keeps a gap of 9 to 14 points. P4, P1 and E2 keep the largest gaps of all, but they only score when a host is judged to have an API or OAuth surface. On 394, 386 and 426 hosts respectively they report info and cost nothing. For hosts where they do apply, a failure is expensive. That is a different reader from the one with a blog.

Three cautions. The band counts a host's own pass on the check being tested, which flatters the passing side slightly. E2 has only nine passing hosts in the band and M4 sixteen, so treat those two as hints. P2 has no passing hosts in the band, which is why it is missing: we cannot estimate a gap from zero observations.

The ladder

Combine three facts: the like-for-like gap, how many hosts are failing, and how much work the fix takes. The first two come from the data. The third is our editorial estimate of effort, not a measurement, so we label it that way.

C1 sits at rung four only because of effort. If you control your server or run a framework with a middleware layer, it moves to rung one: it has the heaviest weight, the most failures and a 13.2-point gap.

Step two: the raw gap lies

The tempting next step is to compare the mean score of hosts that pass a check with the mean of hosts that do not. Here is that comparison for the checks that matter most, and it is nearly useless:

Every row lands between 26 and 37 points. D2, the XML sitemap, shows the biggest raw gap of all, and a sitemap is a file almost every competent site already has. The gap is real, but it is not the sitemap's doing. It exists because a site that publishes a sitemap also tends to have a clean robots.txt, a canonical host and sane headings. Passing one check is a marker of passing many.

The data shows it directly. Hosts that pass D2 pass 8.4 other checks on average; hosts that fail it pass 2.1. Hosts that pass C1 pass 10.1 others, hosts that fail it 5.3. The pass-versus-fail comparison is mostly a comparison of mature sites against neglected ones.

To strip that out, we held maturity roughly constant. We took only hosts that pass between 6 and 10 checks in total, a band of 320 hosts, and repeated the comparison inside it. Hosts in that band look alike: they have the basics and are missing the newer items. Now the gaps separate:

Raw gaps of 26 to 37 collapse to 0.6 to 13.6. Four checks keep a gap above nine points: C2, C1, B2 and D3. D1, D2, C3 and E3 drop to nearly nothing, because in this band almost everyone already passes them. They are table stakes; failing them hurts, and passing them does not distinguish you.

Two cautions on this table. The band counts a host's own pass on the check being tested, so the comparison is slightly generous to the passing side. And E2 has only nine passing hosts in the band, so treat its 6.5 as a hint. P2 has no passing hosts in the band at all, which is why it does not appear: we cannot estimate a gap from zero observations.

The ladder

Combine three facts: the like-for-like gap, how many hosts are failing, and how much work the fix takes. The first two come from the data. The third is our editorial estimate of effort, not a measurement, so we label it that way.

C1 sits at rung four only because of effort. If you control your server or run a framework with a middleware layer, it moves to rung one: it has the largest weight, the most failures and a 13.2-point gap.

Rung one: B2, explicit AI bot rules

B2 asks whether robots.txt names AI crawlers explicitly. A blanket User-agent: * block does not count, because it tells an agent nothing about whether you want it. 65.2% of hosts fall short here, and it is the cheapest fix on the ladder.

Name the crawlers you have a view on, and say allow or disallow for each. A minimal file that allows search-style access and blocks training looks like this:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

According to RFC 9309, the robots protocol, its rules are not a form of access authorization; they declare your wishes. Whether to block or allow is your decision, and the check passes either way as long as the declaration exists. If you are unsure which tokens exist, Every AI Crawler User-Agent in 2026 lists the verified set, and What Sites Tell GPTBot in robots.txt, and Whether They Mean It measured what a sample of sites actually declare and whether the edge enforces it. Note the second post's finding before you rely on a rule: a declaration is a request, and enforcement is a separate layer.

On WordPress, ThinkRank manages robots.txt and robots meta from one place, so the rules do not fight with a second plugin that also writes the file. It imports settings from Rank Math, Yoast, AIOSEO and SEOPress, which means you can switch without retyping what you already declared. Whatever plugin you pick, the thing to avoid is two of them writing robots.txt at once. On Shopify, StoreSEO is the place to start for AI-facing files and schema on a store, because robots.txt on Shopify goes through the theme rather than a file you upload.

Rung two: C2, a valid llms.txt

C2 has the biggest like-for-like gap, 13.6 points, and 387 hosts, 56.8%, fall short. The file is plain Markdown at /llms.txt: an H1 with the project name, which the llmstxt.org specification calls the only required section, then links to the pages you want an agent to read.

# Example Store

> Hand-made ceramics, shipped from Lisbon. Prices in EUR.

## Key pages
- [Shop all products](https://example.com/collections/all): full catalogue
- [Shipping and returns](https://example.com/pages/shipping): delivery times and policy

## Optional
- [About](https://example.com/pages/about)

Two things go wrong often enough to mention. First, links inside the file that return 404; we measured that on our own site in Your llms.txt Is Sending AI Agents to 404s. Second, a soft-200 HTML page at /llms.txt that a lazy check mistakes for the real file. The llms.txt generator builds a valid file, and llms-full.txt: When a Full-Text File Beats a Link List covers the heavier variant.

Be straight about what this file does. Google's own documentation says creating it will neither harm nor help visibility, according to the quote we fetched and verified in Does llms.txt Actually Work in 2026?). The 13.6-point gap is a gap in our score, which rewards the file. It is not a measurement of citations or traffic. We include it on the ladder because it is cheap, the check is real, and the file helps tools that do fetch it. We do not claim it ranks you.

On WordPress, ThinkRank manages llms.txt next to robots.txt, so one plugin owns both files instead of three fighting over them. On Shopify, StoreSEO covers the store side. Either beats pasting a static file once and forgetting it.

Rung three: D3, a Link header that points to discovery files

D3 asks whether your responses carry a Link: header that points an agent at discovery resources. 48.0% of hosts fall short, and 218 of the 320 band hosts already pass, which is why its like-for-like gap, 9.2, is the smallest of the four. It is also one header.

The llms.txt specification added two link relations on 10 August 2026, rel="describedby" and rel="alternate" with type="text/markdown", and they can travel in either an HTML <link> tag or an HTTP Link: header. A header for the homepage might read:

Link: </llms.txt>; rel="describedby"; type="text/markdown"
Link: </sitemap.xml>; rel="sitemap"

On nginx that is two add_header lines; on Cloudflare it is a Transform Rule. Link Headers for AI Agent Discovery walks through each platform. One trap, according to the platform examples in that post: sites that already send a Link: header often spend all of it on preconnect and dns-prefetch hints. Those do not count as discovery, so check the header you have before assuming you are covered.

Rung four: C1, Markdown when an agent asks for it

C1 is the heavy one. It asks whether your server returns text/markdown when a client sends Accept: text/markdown. It carries a weight of 8 or more, the highest in the rubric, and 478 of 681 hosts do not pass. The like-for-like gap is 13.2 points, close to C2's.

You can test it in one command:

curl -sI -H "Accept: text/markdown" https://example.com/ | grep -i content-type

If that prints text/html, you fail. Fixing it is a server job, not a content job. The same page needs a second representation, and the server needs to choose it by header. How to serve text/markdown to AI agents on Next.js, on WordPress, on Hugo and on Astro give the exact change per stack.

Before you start, read Docs Sites and AI Agents: Four Markdown Routes, 52 Sites Tested. It found that documentation sites, a friendly population, still disagree about how to ask for Markdown: header, .md suffix, /index.md and a query string are all in use, and none of the 52 sites answered all four. AIScan's C1 tests the header route and does not probe the .md suffix, so a site that serves only the suffix can fail the check even though the Markdown exists.

Rung five and six: for API owners and stores

P4 (OAuth discovery metadata), P1 (an API catalog under RFC 9727) and E2 (a machine-readable API description) only score when AIScan finds an API or OAuth surface to judge. On most hosts they report info: 394, 386 and 426 of 681. Where they do apply, failing costs 4, 4 and 3 weight points and the like-for-like gap is 13.8 to 17.2 points. If you run an API, put these ahead of D3. OAuth Discovery for AI Agents covers P4, Publish an API Catalog Agents Can Discover covers P1, and an OpenAPI document covers E2.

M4, machine-readable pricing, applies only to the 231 commerce hosts. 202 of them fail or earn partial credit, and the band gap is 13.4 points on thin data (16 passing hosts). Make your pricing machine-readable for shopping agents is the fix, and What a Shopping Agent Can Read About Your Prices measured it on 282 sites. For a Shopify store, StoreSEO is the place to start on structured product data.

Fixes that look big and are not

Some entries in the loss table look expensive and are not worth your afternoon.

D1, D2, E3 and C3 are hygiene. Their like-for-like gaps run 0.6 to 2.8 points. You should still pass them, since a missing sitemap or an empty-looking page is a failure in its own right. Your Sitemap Passes the Check and Still Fails Agents and How Many Sites Show an AI Crawler a Blank Page cover what breaks on those. But if you already pass them, more work there raises nothing.

E1, E4 and E5 show no gap at all. Correct 404 handling, HTTPS with a canonical host, and a content feed come out between -0.7 and -3.8 points inside the band. A negative gap is not evidence that a feed hurts you. It means these checks do not tell similar sites apart in this corpus.

B1, B3, P2, P3 and M3 are early-adopter checks. They report info on most hosts, 563 for B1, 610 for B3, 604 for P2 and P3, and all 231 for M3, so a missing file costs nothing and a pass is a bonus. B1 is Content Signals in robots.txt; P2 is the MCP Server Card. Too few hosts pass in the band to estimate a clean gap for any of them. If you run an MCP server, publish a card: Publish an MCP Server Card shows how. If you run a content site, it is a later step. M3 is a measured zero: the x402 signal earned nothing on all 231 commerce hosts.

What AIScan cannot see

A score gap is not a promise. These are the limits we cannot get around.

• We see what a probe sees. AIScan fetches your pages as a client, from one network location. It cannot tell you whether an agent from a given vendor would be blocked at your edge or served something different. See the declaration census for how often those two differ.
• We cannot see citations. No part of the score measures whether an assistant ever quotes you. The ladder ranks what our rubric rewards. It does not rank what drives traffic from AI products.
• Latest scan only. One scan per host means a host that fixed a check after our last scan still shows as failing. Hosts that scan themselves repeatedly are more likely to be current.
• Selection bias. Everyone in the corpus was scanned by choice. Sites that have never heard of agent readiness are under-represented.
• Observational data. Holding maturity constant removes the largest confounder, not all of them. Hosts that pass C2 may also write better content, run newer stacks or employ someone who read an article like this one.
• Thin cells. E2 has nine passing hosts in the maturity band, M4 has sixteen and P2 has none, so we do not rank them with confidence.
• Applicability. P1, P4 and E2 score only when a probe finds an API or OAuth surface. If our detection misses yours, the check shows info and the ladder says nothing about you.

Run it on your own site

The fastest way to find your position on the ladder is to scan. Run npx aiscan-cli yoursite.com, or paste your URL at aiscan.site, free and without an account. Look at four check IDs in the report: B2, C2, D3 and C1. They map onto the ladder above, and the report gives the exact evidence string for each failure. If you would rather check by hand, the curl command in the C1 section handles that check, curl -s https://yoursite.com/robots.txt shows what B2 reads, and curl -sI https://yoursite.com/ shows your Link: headers for D3.

Audit Your Own Agent Readiness From a Terminal in Ten Minutes shows how to read the output. When you have fixed one rung, scan again, because the order of the ladder can change once you pass the first rung.

Every check here has a reference page: content checks for C1 and C2, bot access checks for B2, discoverability checks for D3 and the hygiene items, and capabilities checks for the early-adopter group. The full set of how-to guides is at /guides.