--- title: "What a Failing AIScan Check Really Says: 2,961 Fail Lines Audited" slug: evidence-string-audit published: 2026-10-16T03:17:27.008097+00:00 updated: 2026-10-16T03:21:03.79025+00:00 author: "AIScan Content Team" author_url: https://aiscan.site/about category: "AI Readiness" tags: corpus study, evidence strings, scan report, agent readiness, check:B2, check:C3, check:C1, check:D1, check:E1, check:P4, check:E3 description: "We audited 2,961 failed AIScan checks across 744 sites. 45.9% of fail lines only restate the verdict, and one B2 line hides three fixes. Know which to trust." url: https://aiscan.site/blog/evidence-string-audit --- A failing check on a scan report is supposed to be an instruction. The site owner reads the line, knows what the scanner asked for, knows what came back, and knows what to change. We audited whether AIScan's fail lines do that. For most checks they do part of it. For a few, one identical sentence is shown to hundreds of sites whose problems have nothing in common. AIScan stores every check result with a status, an evidence string and a remediation. On 16 October 2026 we queried production for the newest site-scope scan of every host on the current rubric: 744 hosts, 2,961 failed checks. Every one of those 2,961 carries remediation advice. The gap is in the evidence, the one line that says what the scanner saw. 45.9% of fail lines restate the verdict ("no sitemap found") without a status code or a measurement. And the single most common fail line, shown on 484 sites, covers three different situations that need three different fixes. This is a product audit as much as a reading guide. It tells you how far to trust each evidence line on your own report, what to check by hand when the line is too thin to act on, and what we are asking the AIScan team to change. ## Quick summary | Question | Measured answer | Section | |---|---|---| | How many fail lines show what the scanner received? | 1,419 of 2,961 (47.9%) quote a status code or response; 184 (6.2%) quote a measurement; 1,358 (45.9%) only restate the verdict | Three kinds of evidence | | Which checks always show the same sentence? | Six of fourteen: B2, C3, D2, D3, E2 and E5 show one string on every failure (1,310 fail lines) | Check by check | | What does the biggest string hide? | B2 shows "0 known AI agent(s) addressed in robots.txt" on 484 sites; robots.txt answered 200 on 365 of them, 404 on 48, and was refused or errored on 71 | One string, three jobs | | Where does a fail line mislead? | C1 evidence names only the Accept probe on 495 of 500 fails, although 183 passes were earned at /index.md | The route that is not shown | | Which fail line is a good model? | E1: it names the probe path, the status and the byte count | Models worth copying | | Is the advice missing too? | No. 2,961 of 2,961 fails carry remediation text | Advice versus evidence | **What the data says:** the scanner tells every failing site what to do, but for about half of the failures it does not show the evidence that would let the owner check the scanner's reading. **For site owners:** treat a thin fail line as a prompt to run one command yourself, not as a finding. **For the AIScan team:** the cheapest improvement is to store the status code on every probe, not only on three checks. ## How we measured it The limits come first, because they decide how much of this you should repeat. Every figure was queried on 16 October 2026 from AIScan's production database. The population is every site-scope scan on rubric 2026.08.2, which has been live since 24 August 2026. We reduced each URL to its host, kept the newest scan per host and excluded `aiscan.site`, whose own scans score 100 and would add nothing to a study of failures. Page-scope scans are excluded. That leaves 744 hosts, and 675 of them (90.7%) have at least one failed check. No individual scanned site is named, and no row-level data was selected: every number below is an aggregate. "Fail" means the stored status is `fail`. Checks that report `partial`, `info` or `pass` are out of scope, so a check that is optional on a site (the commerce and application checks are only scored when they apply) contributes fewer lines than the always-scored ones. We did not audit pass evidence except where a section says so. We read the stored `evidence` string, not the rendered report page. Where the report adds wording of its own, that wording is outside this audit. We group strings after replacing the random probe suffix in E1's path and the byte counts, so that two strings that differ only by a number count as one wording. Word counts in E3 are not collapsed, because the number is the measurement. One more limit: we judge a line by what it says, not by whether the scanner's decision was right. A line can be thin and the verdict correct. [Our determinism audit](/blog/aiscan-check-determinism-self-audit) covers whether the verdicts repeat. ## Three kinds of evidence Read as a user would, an evidence line can do one of three things. It can quote what came back from the site: a status code, a content type, a header. It can quote a measurement the scanner made on the page: a word count, a heading count. Or it can restate the verdict in different words, so that the line adds nothing the status badge did not already say. We sorted all 2,961 fail lines by check, because within a check the wording comes from one template. | Kind of evidence | Checks | Fail lines | Share | |---|---|---|---| | Quotes a response (status code, content type, or "no response") | C1, C2, D1, E1, P1, P4 | 1,419 | 47.9% | | Quotes a measurement from the page | C3, E3 | 184 | 6.2% | | Restates the verdict | B2, D2, D3, E2, E5, M4 | 1,358 | 45.9% | The first group is what you want from a fail line. It says "I asked, and this is what I received". The reader can repeat the request and compare. The third group is the problem. "No RSS, Atom or JSON Feed found at the conventional paths" is the E5 line on 105 sites. It is true, and it does say where the scanner looked in general terms. It does not say whether any of those paths answered 404, 403, 200 with an HTML page, or nothing. A site that returns 403 to every non-browser request and a site that never built a feed show the same words. That matters because a refused probe is common. [In our study of blocked probes](/blog/blocked-probe-share-of-failures), 11.9% of hosts refused at least one probe with a 403, 429 or 5xx, and those refusals counted as failures. That study could only be done on C1, C2 and D1, the checks whose fail lines were rich enough for it. Eight of the fourteen failing checks (B2, C3, D2, D3, E2, E3, E5 and M4) store no status at all, so a refusal on those cannot be seen, by us or by the site owner. ## Check by check Here are the fourteen checks that failed on at least one host, with the number of distinct wordings each one shows. | Check | What it looks for | Fails | Distinct wordings | Most common wording's share | |---|---|---|---|---| | B2 | Explicit rules for named AI agents in robots.txt | 484 | 1 | 100% | | C1 | Markdown version of the page | 500 | 30 | 36.6% | | C2 | llms.txt | 348 | 11 | 75.3% | | C3 | Title, h1, meta description, JSON-LD on the homepage | 61 | 1 | 100% | | D1 | robots.txt | 119 | 10 | 40.3% | | D2 | Sitemap file | 186 | 1 | 100% | | D3 | Link header or link tag for discovery | 287 | 1 | 100% | | E1 | Correct 404 on a missing page | 78 | 4 | 87.2% | | E2 | OpenAPI document | 187 | 1 | 100% | | E3 | Server-rendered text and headings | 123 | 25 | 22.0% | | E5 | RSS, Atom or JSON feed | 105 | 1 | 100% | | M4 | Pricing page or JSON-LD Offer | 109 | 3 | 72.5% | | P1 | API catalog at /.well-known/api-catalog | 171 | 5 | 94.7% | | P4 | OAuth discovery documents | 203 | 7 | 88.2% | Six checks, B2, C3, D2, D3, E2 and E5, show exactly one sentence on every failure: 1,310 failures that read identically within their check. That is not a bug by itself. A scanner that looked for a file and did not find it has little else to say. The question is whether a second sentence would let the owner tell two different causes apart, and the next three sections show that for B2 and C3 it would. The wide spread in C1 (30 wordings) and E3 (25) looks like detail but is mostly formatting. C1's 30 wordings are a status code crossed with a content-type spelling, so `text/html; charset=utf-8` and `text/html; charset=UTF-8` count as two. The three most common C1 wordings are the same fact: the server answered 200 with an HTML page when asked for Markdown. E3's 25 wordings are mostly word counts. Variety in the string is not the same as information in the string. ## One string, three jobs B2 is the most common failure in the corpus, and its evidence is the shortest possible: "0 known AI agent(s) addressed in robots.txt". It appears on 484 sites, 100% of B2 failures. The sentence is accurate. It also has at least three causes, each with a different fix: 1. The site has a robots.txt, and it says nothing about named AI crawlers. The fix is to add rules to a file that exists. 2. The site has no robots.txt. The fix is to create one, and to put the rules in it from the start. 3. The scanner could not read robots.txt: the request was refused, rate limited or errored. The fix is on the server or CDN side, and writing AI rules will not change the result. The scan stores a separate robots.txt result in D1, which lets us split the 484 B2 failures by what D1 recorded for the same host: | What D1 recorded for robots.txt | B2 failures | Share | The real job | |---|---|---|---| | Answered 200 | 365 | 75.4% | Edit the file | | Refused, rate limited, error or "no response" | 71 | 14.7% | Stop the block; the rules cannot be read yet | | Answered 404 | 48 | 9.9% | Create the file | Three quarters of B2 failures are the first case, which is the case the B2 remediation ("Add explicit allow/deny rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.") fits. For the other quarter the advice does not fit the cause. Of the 71, 45 are 403 or 429, 17 are 52x server errors, five are a 202, and the rest are a 402, a failed 200 or no response. A site owner in that group can add the rules, scan again, and see the same sentence. One caution about the 365: a 200 from robots.txt is not proof of a real robots file, because a server that answers 200 to every path can serve an HTML page there. [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309), the Robots Exclusion Protocol, defines an unavailable robots.txt (a 4xx) as permission to crawl anything, and says nothing useful about an HTML page served with a 200. We did not parse bodies, so we do not split the 365 further. If your report fails B2, the first move is to know which row of that table you are in. [AIScan](https://aiscan.site/) already stores the answer in the D1 line of the same report, so read D1 next to B2: a `200 OK` with a byte count means edit, a `404` means create, anything else means look at what is blocking the request. The [bot access check guide](/docs/checks/bot-access) explains the rules themselves, and [our robots.txt starter](/blog/robots-txt-starter-for-ai-crawlers) is a file you can paste in. On WordPress, [ThinkRank](https://thinkrank.ai) writes robots.txt, the sitemap and the AI rules from one plugin, so that three plugins do not each write their own version of the file. If you want to check by hand instead: ```bash curl -sIL -A "Mozilla/5.0" https://yoursite.com/robots.txt | grep -iE '^(HTTP|content-type|content-length)' curl -s -A "Mozilla/5.0" https://yoursite.com/robots.txt | grep -iE 'GPTBot|ClaudeBot|PerplexityBot|Google-Extended' ``` The first line gives you the status. The second tells you whether any named agent is addressed. The scan's answer to "0 known agents" is the second command returning nothing. ## The sentence that fits a refusal and an empty page C3 checks the homepage for a title, an h1, a meta description and JSON-LD. It failed on 61 hosts, and all 61 show the same line: "h1: 0, title: false, meta description: false, JSON-LD: false". Look at what that means. A homepage with none of four basic elements is rare among real sites. The line is what the scanner would write if it received a page with nothing on it, and also what it would write if it received a block page, a challenge or an error body. We cannot read response bodies, because AIScan does not store them, but we can look at what the C1 probe recorded for the same host: | C1 result on the same 61 hosts | Hosts | Share | |---|---|---| | 401, 403, 429, 5xx or no response | 40 | 65.6% | | 202 or 203 (accepted, no page yet) | 7 | 11.5% | | 402 or 404 | 2 | 3.3% | | 200 | 12 | 19.7% | Put together, 49 of the 61 hosts (80.3%) did not get a normal page back on the C1 request. On those hosts the C3 failure is most likely the scanner reading a refusal as a homepage. For the 12 that answered 200, the empty-looking result may be real: a page that builds its content in the browser would also produce it, and [our render-blindness study](/blog/empty-dom-render-blindness-corpus) counted those. We cannot say from stored evidence which of the 61 is which, and that is the point. The same sentence is shown to a site that is down for scanners and a site with an empty document. The first needs the CDN rule fixed. The second needs the head tags written. A fail line that said "homepage response was 403" would have sent the owner to the right place. The check by hand is quick: ```bash curl -sL -A "Mozilla/5.0" -o /tmp/home.html -w "%{http_code} %{content_type} %{size_download} bytes\n" https://yoursite.com/ grep -ciE ', 0

" is a fact the reader can check. It does not state the threshold the check applies, so a site that reads "27 words" cannot tell how far it is from passing. Printing the threshold next to each number would cost one clause. ## Advice versus evidence It would be easy to read all of this as "the scanner does not explain itself". The data says something narrower. All 2,961 failures carry remediation text, and the text is specific. B2's says to add explicit allow and deny rules for GPTBot, ClaudeBot, PerplexityBot and Google-Extended. D2's says to publish `/sitemap.xml` and reference it in robots.txt. E2's says to publish an OpenAPI 3.1 document at `/openapi.json` and reference it from `/.well-known/api-catalog` as a `service-desc` link. What the report does not do is connect the two. The advice is written for the most common cause, and the evidence does not say whether the most common cause is the one in front of the reader. For a check where the cause is nearly always the same, such as D3 (no Link header or tag), that is fine. For B2 and C3 it is not, because the share that the standard advice does not fit is a quarter and a majority respectively. There is a second reason the evidence matters beyond the individual site. Evidence also travels. When someone pastes a report into a chat and asks what to do, the assistant has the check ID, the status and the evidence string to work from. A thin line gives it little to go on, and a line that says "robots.txt → 404" gives it the cause. ## Reading your own report Here is how to read a fail line once you know what it can and cannot say. | Check | If the line says | It can also mean | Look at, or run | |---|---|---|---| | B2 | 0 known AI agent(s) addressed | robots.txt missing, or blocked | Read D1 beside it | | C3 | h1: 0, title: false, meta description: false, JSON-LD: false | Scanner was refused or served a block page | Curl the homepage, check the status | | C1 | Accept: text/markdown → 200 (text/html) | `/index.md` was tried too, or not | Curl both routes | | D2 | No /sitemap.xml or /sitemap_index.xml resolved | A sitemap at another path, or a refused request | Check robots.txt for a `Sitemap:` line | | D3 | No Link header or tag | Nothing else; the check is a plain absence | Add the header or tag | | E2 | No OpenAPI document found | A spec at an unlisted path | Check your docs for a spec URL | | E5 | No feed at the conventional paths | A feed at an unlisted path | Check `` for `rel="alternate"` | | P4 | auth-server: 200, protected-resource: 200 | Documents served but invalid | Validate the JSON against the standard | The pattern in the second column is that every "can also mean" is either a refusal or a path the scanner does not try. Both are things the site owner knows about and the report does not. If you are working through a long red list, start with the lines that quote a status, because they are the ones you can trust to show what happened. Then use [the ranking of highest-impact fixes](/blog/agent-readiness-highest-impact-fix) to decide order. For the thin lines, spend the thirty seconds on the matching command first. A wrong fix to a refused probe costs more than the command does. ## What AIScan cannot see This audit has three blind spots of its own. The scan records one evidence string per check, not the full request log. We cannot see the headers the scanner sent or the body it received, so every statement about what a thin line "could mean" is an inference from other stored fields on the same host, mainly the D1 and C1 lines. Where we split a population we say which field we used. The D1 split of B2 assumes that D1's evidence describes the robots.txt response. That matches how [our earlier work](/blog/check-co-pass-matrix) reads D1, and its pass lines look like short robots files, but we did not fetch the files again. And fail lines are the only evidence we audited. Pass and partial lines may be better or worse. A site owner who is trying to decide whether a pass is solid has a related question, and it is covered by [the soft-404 findings](/blog/soft-404-agent-discovery-problem) for the case where it matters most. Product notes from this work, filed as a site-issue request for the AIScan team: - Store the HTTP status for every probe, including B2, C3, D2, D3, E2, E3, E5 and M4. That alone would let those checks separate "absent" from "refused". - B2 should say whether robots.txt was read and from what status, or point to D1. - C3 should name the response status it received for the homepage, and say "homepage fetch refused" when it was. - C1 should list every route it tried on a failure, including `/index.md`. - P4 and D1 should say why a 200 failed. - E3 should print the threshold next to the count. ## Method Every figure came from one pattern: take the newest site-scope scan per host on rubric 2026.08.2, exclude `aiscan.site`, unnest `result->'checks'`, keep the `fail` rows and group by check and by evidence string. ```sql with latest as ( select distinct on (host) host, result from (select lower(regexp_replace(regexp_replace(url,'^https?://',''),'/.*$','')) host, result, created_at from scans where rubric_version = '2026.08.2' and scope = 'site') s where host not like '%aiscan.site%' order by host, created_at desc), f as ( select c->>'id' id, regexp_replace(regexp_replace(c->>'evidence', 'aiscan-404-probe-[a-z0-9]+','P'),'[0-9]+ bytes','N bytes') ev from latest, jsonb_array_elements(result->'checks') c where c->>'status' = 'fail') select id, count(*) fails, count(distinct ev) wordings, round(100.0 * max(n) / sum(n), 1) top_share from (select id, ev, count(*) n from f group by id, ev) g group by id order by id; ``` The B2 split joins the B2 status to the D1 evidence for the same host and classifies D1 as "HTTP 404", a string starting `200 OK`, or anything else. The C3 split joins C3 failures to the C1 evidence on the same host. The kind-of-evidence table assigns each check to one group by reading its wording: C1, C2, D1, E1, P1 and P4 quote a response; C3 and E3 quote a measurement; the rest restate the verdict. Fourteen "no response" lines (five in C1, four in C2, two in D1, three in E1) are counted with the response group because they report what the probe received. ## Run it on your own site Paste your address into [aiscan.site](https://aiscan.site/) or run `npx aiscan-cli yoursite.com`, then read the failing lines against the table above. B2 and D1 go together, C3 and C1 go together, and E1 is the one to trust. The [bot access](/docs/checks/bot-access), [content](/docs/checks/content) and [discoverability](/docs/checks/discoverability) check guides explain each fix, and the [guides](/guides) list a walkthrough for every check. For the status codes themselves, [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110#name-status-codes) is the reference: a 202 means accepted but not finished, so a scanner that receives one has not yet seen your page. Scan again after the change, and read the evidence line before the badge.