--- title: "What Happens When a Site Is Scanned Again: 616 Hosts, 170 Improved, 73 Got Worse" slug: rescan-score-changes-corpus published: 2026-11-06T03:09:13.014337+00:00 updated: 2026-11-06T03:09:13.014337+00:00 author: "M Asif Rahman" author_url: https://ar.bd category: "AI Readiness" tags: check:B2, check:C1, check:C2, check:D3, check:E1, corpus study, rescan, AI readiness description: "Of 616 rescanned sites, 170 scored higher, 73 lower and 373 the same. Which checks flip to pass, which regress, and how the starting score matters. Own scan data, 6 Nov 2026." url: https://aiscan.site/blog/rescan-score-changes-corpus --- Of 616 hosts in our scan corpus that were scanned more than once, 170 scored higher on their latest scan than on their first, 73 scored lower, and 373 did not move. The median change is 0 points and the mean is +4.2. Most of the gain comes from sites that started low, and the checks that flip to pass most often are the ones with a clear file to publish. ## Quick summary | Question | Answer | Source | |---|---|---| | How many rescanned hosts improved? | 170 of 616 (27.6%) | AIScan scan database, rubric 2026.08.2, 6 Nov 2026 | | How many got worse? | 73 of 616 (11.9%) | same | | How many did not change? | 373 of 616 (60.6%) | same | | Median and mean change | 0 and +4.2 points | same | | Moved up 10 points or more | 101 hosts (16.4%) | same | | Moved down 10 points or more | 36 hosts (5.8%) | same | | Mean score, first scan to latest | 52.9 to 57.1 | same | | Mean gap between first and latest scan | 21.2 days | same | | Check that flips to pass most often | C1, Markdown content negotiation (63 hosts) | same | | Check that most often drops out of pass | B1, Content Signals in robots.txt (38 hosts) | same | **Short answer for a site owner:** a rescan after a real fix usually shows the fix. A rescan with no changes shows the same score. Hosts that started under 40 gained 8.4 points on average, and hosts that started at 80 or more gained 0.5. If your score fell, look at the checks that depend on your server answering correctly (robots.txt, sitemap, HTTPS, 404 handling) before you look at anything you published. Run `npx aiscan-cli yoursite.com` (or paste the address at [aiscan.site](https://aiscan.site/), free, no account), note the score, change one thing, and scan again. If you would rather compare by hand, the `curl` commands are near the end. ## What this study measures, and what it cannot We took every host with at least two site-scope scans under rubric 2026.08.2, at least one hour apart, and compared its first scan with its latest. `aiscan.site` is excluded, and no scanned host is named in this post. There are 616 such hosts. This is a self-selected group. People rescan because they scanned, read a report and probably changed something. The numbers below describe what happens to sites whose owners came back, not what happens to the web. A site nobody rescans is not in this table at all. Three more limits: - **Two points in time.** A host scanned five times appears once, as first versus latest. A fix made and reverted between those scans is invisible. - **Status, not cause.** The stored report says a check moved from partial to pass. It does not say whether the owner changed something, a CDN rule changed, or the site had an outage during the first scan. - **Rubric version.** Only scans under 2026.08.2 are compared, so a score change never comes from a rule change. ## How much scores move | Outcome | Hosts | Share | |---|---|---| | Up | 170 | 27.6% | | Down | 73 | 11.9% | | Unchanged | 373 | 60.6% | | Up by 10 or more | 101 | 16.4% | | Down by 10 or more | 36 | 5.8% | | Largest gain | +73 | | | Largest drop | -85 | | *Source: AIScan scan database, rubric 2026.08.2, 616 rescanned hosts, queried 6 Nov 2026.* The distribution is lopsided but not one-way. Gains outnumber drops by 97 hosts, and the mean is positive, but the median is zero because six in ten rescans find the same score. A drop of 85 points is not a rounding artefact. Something that used to answer stopped answering. ## Where you start decides how much you gain | First score | Hosts | Up | Down | Mean change | Up by 10+ | |---|---|---|---|---|---| | 0 to 39 | 168 | 45 | 11 | +8.4 | 31 | | 40 to 59 | 215 | 57 | 31 | +4.4 | 36 | | 60 to 79 | 148 | 42 | 21 | +1.1 | 24 | | 80 and above | 85 | 26 | 10 | +0.5 | 10 | *Source: same, 6 Nov 2026. Bands use the first scan's score.* Low starters have the most to gain, and they gain it: 31 of 168 hosts under 40 climbed 10 points or more. The 80-and-above group still has 26 hosts that went up, but their mean is half a point because their remaining gains are small and their possible losses are not. The band from 40 to 59 is the one to watch. It has the most hosts and the most drops (31). Its mean gain is positive, but one host in seven in that band scored lower on the latest scan. ## Does waiting longer help? | Gap between scans | Hosts | Up | Down | Mean change | |---|---|---|---|---| | Under 1 day | 11 | 2 | 1 | +6.0 | | 1 to 7 days | 200 | 34 | 11 | +3.8 | | 7 to 30 days | 181 | 54 | 20 | +4.1 | | 30 days or more | 224 | 80 | 41 | +4.5 | *Source: same, 6 Nov 2026.* Barely. The mean change sits between +3.8 and +4.5 across the three bands that hold most hosts. What changes with a longer gap is the spread: the 30-days-or-more group has the highest share of ups (35.7%) and the highest share of downs (18.3%). Time gives a site more chances to improve and more chances to break. The under-one-day group is 11 hosts, too few to read. A fast rescan is not a worse test than a slow one. Of the 200 hosts rescanned within a week, 34 gained, which is a fix applied and checked the same week. ## Which checks flip to pass For every host, we compared each check's status on the first scan with its status on the latest. "Flipped to pass" means not pass first, pass later. "Dropped out of pass" means the reverse. "Stayed pass" means pass both times. | Check | Name | Flipped to pass | Dropped out of pass | Stayed pass | |---|---|---|---|---| | C1 | Markdown content negotiation | 63 | 7 | 144 | | B2 | Explicit AI bot rules | 61 | 26 | 147 | | C2 | /llms.txt | 57 | 7 | 251 | | D3 | Link header for discovery | 48 | 11 | 296 | | E2 | Machine-readable API description | 43 | 0 | 59 | | C3 | Structured HTML | 39 | 10 | 287 | | P4 | OAuth discovery metadata | 38 | 5 | 79 | | P1 | API Catalog (RFC 9727) | 38 | 6 | 96 | | E4 | HTTPS and canonical host | 27 | 20 | 347 | | E3 | Heading hierarchy and server-rendered text | 27 | 10 | 407 | | D1 | robots.txt present and sane | 26 | 13 | 423 | | B1 | Content Signals in robots.txt | 23 | 38 | 73 | | D2 | XML sitemap | 21 | 17 | 445 | | E1 | Correct 404 handling | 21 | 19 | 444 | *Source: same, 616 hosts, 6 Nov 2026. Checks with fewer than 20 flips in both directions (M4, E5, P3, P2, B3, M1, M3) are omitted.* Three patterns stand out. **The big flips are files and headers.** C1, C2, D3, P1, P4 and E2 are all things a site owner publishes or configures once: a markdown response for `Accept: text/markdown`, an `/llms.txt`, a `Link` header, an API catalog, OAuth metadata, an OpenAPI description. Their drop-out counts are small (0 to 11), so once published they mostly stay. The [capabilities checks page](https://aiscan.site/docs/checks/capabilities) explains each one, and the [agent discovery files census](https://aiscan.site/blog/well-known-agent-files-corpus) shows how rare they were in the first place. **B2 moves in both directions.** 61 hosts gained an explicit AI bot rule and 26 lost one. B2 reads robots.txt, so it moves when somebody edits that file, when a CMS plugin rewrites it, or when the file stops being served. Because the stored evidence does not keep the file body, we cannot say which. **B1 is the one that shrinks.** More hosts dropped out of B1 (38) than flipped into it (23). The Content Signals line is the newest robots.txt convention in the rubric, and a robots.txt that gets regenerated by a plugin or a deploy loses a line it did not know about. If you add a Content Signals line, rescan after your next deploy to confirm it survived. ## What the score drops have in common | Measure | Hosts | |---|---| | Hosts that scored lower | 73 | | Lost 3 or more passes | 23 | | Lost passes and also gained some | 21 | | Hosts that scored higher but also lost a pass | 32 | *Source: same, 6 Nov 2026. A lost pass means a check that was pass on the first scan and not on the latest.* Of the 73 hosts that scored lower, 23 lost three or more passes in one go. That is the signature of a site-level change, not a single edit: a robots.txt that returns an error, a host that began redirecting, a page that now needs JavaScript. The checks that most often leave the pass column are B1 (38), B2 (26), E4 (20), E1 (19) and D2 (17). Most of those five depend on how the server answers, not on content you wrote. The [soft-404 check](https://aiscan.site/docs/checks/discoverability) (E1) and the sitemap check (D2) are the two to read first after a hosting change. For how a status line is worded and why "pass" and "partial" can look the same in stored evidence, see [the evidence string audit](https://aiscan.site/blog/evidence-string-audit). ## Gains come from several checks at once | Measure | Hosts | |---|---| | Hosts that scored higher | 170 | | Gained 3 or more passes | 75 | | Scored higher with no check flipping to pass | 18 | | Scored higher but also lost a pass | 32 | *Source: same, 6 Nov 2026.* 75 of the 170 improvers flipped three or more checks to pass. Gains tend to come in bundles, so the files that flip most often (`/llms.txt`, a `Link` header, a markdown response) probably get added together, though the stored report cannot confirm that. The 18 hosts that went up with no check flipping to pass gained through partial credit, which the pass count does not show. For a view of which checks tend to pass together, see the [check co-pass matrix](https://aiscan.site/blog/check-co-pass-matrix). For how platforms compare, see the [platform league table](https://aiscan.site/blog/platform-league-table-agent-readiness). ## What to do with this 1. **Scan, change one thing, scan again.** The data supports fast rescans: 34 of the 200 hosts rescanned within a week gained. 2. **Fix the files first.** C1, C2, D3, P1, P4 and E2 flip to pass often and rarely come back out. 3. **Rescan after every deploy and every plugin update.** The checks that regress are the ones a deploy can silently change: B1, B2, E4, E1 and D2. 4. **Do not chase 100 when you are at 85.** The 80-and-above group gains half a point on average, and a lost pass costs more than a new one earns. 5. **Read a drop as a server question first.** A 20-point fall with three lost passes points at the host, not your content. If you would rather check the regression-prone checks by hand: ```bash # Is robots.txt served, and as text? curl -s -o /tmp/robots.txt -w '%{http_code} %{content_type}\n' https://yoursite.com/robots.txt # Does it still declare your sitemap and AI bot rules? grep -i -E 'sitemap|gptbot|claudebot|content-signal' /tmp/robots.txt # Does a path that cannot exist return 404? curl -s -o /dev/null -w '%{http_code}\n' https://yoursite.com/zz-not-a-real-page ``` The first line should read `200 text/plain`. The grep should print your sitemap line and your bot rules. The last command should print `404`. If it prints 200, your host answers every path with a page, and that is the E1 failure. The robots.txt format is defined in [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309.html). ## What AIScan cannot see here - **Cause.** Stored reports record statuses and evidence strings, not what the owner changed. - **File bodies.** We cannot say what a robots.txt or llms.txt contained, only how the check graded it. - **Hosts that never came back.** The study contains only sites that were scanned at least twice. - **The middle of the story.** A host scanned five times is compared only at its ends. - **Demand context.** Vendors in this space advise rescanning after changes and publish expected gains of 15 to 20 points for robots.txt and llms.txt work (one such guide, retrieved 6 Nov 2026). We could not find an independent measurement. This post is one, on one rubric, with the self-selection above. ## Your next action Run the scan, write down the score and the lines that are not pass, change one thing, and run it again. Read the capabilities and discoverability lines first. Details for every check are on the [capabilities page](https://aiscan.site/docs/checks/capabilities), and the walkthroughs are collected at [aiscan.site/guides](https://aiscan.site/guides). ## Sources and dates - AIScan scan database, rubric 2026.08.2, first and latest site-scope scan per host, 616 hosts with scans more than one hour apart, `aiscan.site` excluded, queried 6 Nov 2026. - [RFC 9309, Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html). - Related AIScan studies: [Agent Discovery Files Census](https://aiscan.site/blog/well-known-agent-files-corpus), [Check co-pass matrix](https://aiscan.site/blog/check-co-pass-matrix), [Evidence string audit](https://aiscan.site/blog/evidence-string-audit), [State of AI agent readiness 2026](https://aiscan.site/blog/state-of-ai-agent-readiness-2026).