Gate AI readiness in CI so a regression fails the build

Table of contents

Quick summary
Why a passing CI check can mean nothing
The control path makes the gate honest
What ten live hosts returned
Path one: gate a deployed URL with the AIScan CLI
Path two: gate the build output before it ships
Platforms with no build step: monitor instead
Reading the exit codes and the JSON
Where AIScan fits, and where it doesn't
Wire the control path into your next commit

A green pipeline is supposed to mean the site is fine. On an AI readiness check it often means something narrower: that the gate could not tell a file from a phantom. This step, which appears in a lot of pipelines, is weaker than it looks.

curl -f https://yoursite.com/llms.txt      # exits 0. Proves nothing.

Exiting 0 proves the server answered 200. It does not prove the file exists. On several hosting platforms a request for a path nobody ever created also returns 200, with the application shell as the body, so a gate written that way passes on the day the file is deleted and every day after.

This guide builds two gates that survive that. One runs the scanner against a deployed URL. One runs against your build output before anything ships. Both are short, both are copy-paste, and the second needs no scanner at all.

Quick summary

Question	Short answer
What breaks a naive gate?	A missing path returning HTTP 200 and the app shell instead of a 404
How do you catch it?	Fetch a path you know does not exist and compare it against the file you care about
Fastest gate	`npx aiscan-cli yoursite.com --min-score 85 --fail-on essential`
Exit codes	0 pass, 1 gate failed, 2 bad usage, 3 network or API error
Which checks it covers	D1 robots.txt, C2 llms.txt, C3 and E3 server-rendered HTML, E1 correct 404
Gate before deploy or after?	Before, if your platform produces a build folder. After, if it does not
Platforms with no build step	WordPress, Shopify, Wix, Squarespace: schedule a scan instead
What the gate cannot see	Per-crawler policy, paywalls, and anything your CDN does to a bot but not to you

Why a passing CI check can mean nothing

Single-page applications are usually served by a catch-all route. Anything the router does not recognise gets the shell, because that is how client-side routing works: the browser loads the shell, reads the URL, and draws the right view. It is correct for pages. It is wrong for files, and a crawler asking for /llms.txt gets HTML with a 200 attached.

We measured this on a live Replit app. Verified on 9 September 2026, askquran.chat/llms.txt returns HTTP 200, text/html, 5,712 bytes. A path invented for the test returns HTTP 200, text/html, 5,712 bytes. Both bodies hash to the same MD5, reproduced three times over ten minutes, and both begin <!DOCTYPE html>. There is no llms.txt on that host. A pipeline asserting a 200 would have gone green every run since the app was created.

What makes the case sharp is the same host's /robots.txt: a real 7,548-byte text/plain file, served correctly. One file is genuine and one is a mirage, on one origin, and a status-code gate scores them identically. That behaviour comes from the deployment target rather than the code, which the Replit setup guide covers in more detail.

The control path makes the gate honest

The fix is one extra request. Before you assert anything about a file, ask the server for a path you are certain does not exist, and fail the build if that returns 200.

CTRL=$(curl -s -o /dev/null -w '%{http_code}' \
  "https://yoursite.com/no-such-file-$RANDOM.txt")
[ "$CTRL" = "404" ] || { echo "control path returned $CTRL, gate is blind"; exit 1; }

Run that first and every later assertion means something. Skip it and none of them do. The same comparison works on byte counts when a platform returns a real 404 page that happens to be large: fetch both, compare %{size_download}, and treat identical sizes as identical bodies.

This is check E1 in the rubric, and it is worth understanding why a scanner grades it at all. A crawler discovering your site has no map of which paths are real. It probes conventional locations, and a soft 200 tells it every one of them exists. The site then looks like it publishes a full discovery layer while publishing none of it, and the agent that trusted the answer gets HTML where it asked for a plain-text file. A correct 404 is the only signal that separates your real files from your router.

What ten live hosts returned

Ten origins, two requests each, browser user agent from a datacentre address, all fetched from this container on 9 September 2026.

Origin	llms.txt	Invented path	Verdict
aiscan.site	200, 22,831 B	404	Real 404, gate is safe
Next.js docs	200, 12,724 B	404	Real 404, gate is safe
Wix	200, 36,670 B	404, 2,501 B	Real 404, gate is safe
Squarespace	200, 2,088 B	404	Real 404, gate is safe
OpenTelemetry (Hugo)	200, 2,151 B	404	Real 404, gate is safe
Cypress docs (Docusaurus)	200, 69,997 B	404	Real 404, gate is safe
Hugo docs	404	404	No file, control honest
Docusaurus docs	404	404	No file, control honest
`askquran.chat` (Replit)	200, 5,712 B	200, 5,712 B	Soft 200, gate is blind
`visa4.travel` (Replit)	200, 3,904 B	200, 11,484 B	Soft 200, gate is blind

Two of ten, both on the same hosting platform. That matches the wider sweep we ran in September, where 10 of 12 published Replit apps returned 200 for a path that did not exist, against 15 of 15 Hugo sites and 15 of 15 Docusaurus sites returning a genuine 404. The risk is concentrated, not universal, and one command tells you which side you are on.

Path one: gate a deployed URL with the AIScan CLI

The scan is the recommended route because it answers the whole question in one request set rather than one file at a time. It is free, needs no account, and reports D1, D2, B1, B2, C1, C2, C3, E1, E3 and E5 with the evidence it observed for each.

Try it locally first: npx aiscan-cli yoursite.com. Node 18 or later, no install, no dependencies.
Read the score and note which checks are essential tier. Those are the ones worth failing a build over.
Pick a floor slightly under today's score so the gate catches regressions rather than blocking on work you have not done yet.
Add the workflow below to .github/workflows/agent-readiness.yml.
Push, then read the job summary on the run page.

name: Agent readiness
on:
  push:
    branches: [main]
  schedule:
    - cron: '0 6 * * 1'
jobs:
  aiscan:
    runs-on: ubuntu-latest
    steps:
      - name: Control path must 404
        run: |
          CTRL=$(curl -s -o /dev/null -w '%{http_code}' \
            "https://yoursite.com/no-such-file-$RANDOM.txt")
          echo "control path: $CTRL"
          [ "$CTRL" = "404" ] || exit 1
      - name: Scan
        run: npx aiscan-cli https://yoursite.com --min-score 85 --fail-on essential --md >> $GITHUB_STEP_SUMMARY

Two details in there are worth knowing rather than copying blind. GITHUB_STEP_SUMMARY is, in GitHub's own words, "the path on the runner to the file that contains job summaries from workflow commands", so the --md report renders on the run page instead of being buried in a log. And according to GitHub's events reference, scheduled workflows "run in UTC", "run on the latest commit on the default branch", and the shortest interval available is once every five minutes.

Two runner problems have easy answers. If npm is missing from your image, the CLI documents a fallback that needs only Node: curl -fsSL https://aiscan.site/cli.mjs | node - example.com. And according to the CLI's own documentation, results are cached for five minutes, which bites the moment you fix something and immediately re-run the job: the gate re-reads the pre-fix result and fails again. Add --fresh to any step that runs after a deploy in the same pipeline.

Path two: gate the build output before it ships

If your platform produces a folder, you can fail the build before anyone sees it. This path uses no scanner and no network, so it also works on a private repository with no deploy preview.

Build as normal, then assert the files exist on disk rather than over HTTP. test -s public/llms.txt fails on a missing file and on an empty one.
Assert the page shipped words. For Docusaurus, grep -c 'id="__docusaurus"></div>' build/index.html returns 1 exactly when that route emitted an empty React root, which is the whole of a C3 and E3 failure in one line.
For any framework, strip the tags and count. Under about 100 words on a real content page means the HTML went out without a body.
Fail the job on either condition.

set -e
test -s public/llms.txt
test -s public/robots.txt
W=$(perl -0777 -pe 's/<(script|style|noscript|svg)\b.*?<\/\1>//gsi;
                    s/<[^>]+>/ /gs' public/index.html | tr -s '[:space:]' ' ' | wc -w)
echo "rendered words: $W"
[ "$W" -ge 100 ] || { echo "index.html shipped no body"; exit 1; }

That word-count command matters more than it looks. The one-liner most guides print, sed 's/<[^>]*>/ /g', cannot strip a multi-line <script> block: measured against the Next.js site it returned 626 words where the correct command returned 1,041. A gate built on the broken version passes pages that shipped nothing.

Checking out the repository first needs actions/checkout. Fetched from the action's own repository on 9 September 2026, the current major is v7.

Platforms with no build step: monitor instead

There is no honest way to gate a hosted builder pre-deploy. There is no build folder, no deploy step to hook, and on Wix the only code path is a site function answering at <baseUrl>/_functions/<functionName>. The correct answer is a scheduled scan, plus a plugin or app that keeps the files right in the first place.

Platform	Pre-deploy gate	What to do instead
Hugo, Docusaurus, Astro, Next.js	Yes, assert on the build folder	Add the scan as a second gate on the deployed URL
Replit, Lovable	Partly, depends on deployment target	Control path first, always
WordPress	No	Weekly scheduled scan, files managed by a plugin
Shopify	No	Weekly scheduled scan, files managed by an app
Wix, Squarespace, Webflow, Framer	No	Weekly scheduled scan only

A scheduled scan needs no repository and no build. The schedule block from the workflow above runs on its own with the push trigger removed, so a WordPress or Shopify site can have a weekly gate that opens an alert when the score drops, without a single line of application code changing hands. Point it at a deep content page as well as the root: a route that ships an empty body is a per-route outcome, not a site-wide one, and scanning only the homepage misses it.

On WordPress, ThinkRank is the recommendation, for a reason specific to this problem. One plugin owns robots.txt, the robots meta tags, schema, the sitemap and llms.txt together, which removes the exact regression a gate exists to catch: three SEO plugins each writing their own robots.txt and whichever saved last deciding what crawlers see. Setup reads your existing configuration straight out of Rank Math, Yoast, All in One SEO or SEOPress, so nothing is retyped. Rank Math and Yoast both go deeper on keyword and readability analysis, and if that is where your editing hours go they stay the better daily tool. Neither writes an llms.txt at all.

Shopify has the same shape, and StoreSEO is the equivalent answer there. Its llms.txt is built from the store's live catalogue, collections, pages and articles rather than typed once and left to drift behind a moving product list, and an agents.md editor sits beside it. The theme-level route, robots.txt.liquid, stays yours whatever app you add.

Reading the exit codes and the JSON

The CLI documents four exit codes and all four were verified on 9 September 2026 against live sites: 0 pass, 1 gate failed, 2 bad usage, 3 network or API error. Scanning aiscan.site with --fail-on essential exited 0. Scanning the Hugo documentation site with the same flag exited 1, because it fails D1 and B2 at essential tier, and --min-score 85 against its score of 41 exited 1 as well.

For anything more specific than a floor, take the JSON. --json returns overallScore, level, rubricVersion, shareUrl and a checks array in which every entry carries id, dimension, tier and status. That is enough to fail on one named check:

npx aiscan-cli yoursite.com --json \
  | jq -e '[.checks[] | select(.id=="C2" and .status=="pass")] | length > 0'

Where AIScan fits, and where it doesn't

The scan reads your site as an ordinary client. It cannot see what your CDN does to a crawler that is not us, which is the gap that matters most in 2026: on 40 measured hosts a robots.txt Disallow was backed by an actual edge refusal only 45% to 67% of the time, and a paywall or a 402 in front of an AI crawler is invisible to every scanner in the field, ours included. It also grades declarations rather than enforcement across the whole bot-access dimension. A gate tells you your files are correct. It does not tell you a crawler was allowed to read them.

You can also run every check here by hand. The control path is one curl, the file assertions are test -s, and the word count is the command above. The scan is faster and reports evidence per check, but nothing in this guide requires it.

Wire the control path into your next commit

Start with the cheapest half: add the control-path step to one workflow today and see what it returns. If it comes back 200, that is your finding, and every other assertion in that pipeline was decorative.

Then scan the deployed URL with npx aiscan-cli yoursite.com, or paste it at aiscan.site, and read E1 first, then D1, C2, C3 and E3. Those checks are documented on the discoverability reference page and the content reference page; every CLI flag is listed under the CLI docs; and the platform walkthroughs are collected at /guides. If your stack emits a build folder, the Docusaurus guide has the one-line render assertion, and why React sites go out empty explains what the gate is protecting you from.

Frequently asked questions

My CI step returns HTTP 200 for llms.txt but the file is not on the server. How is that possible?

Your host is serving a catch-all route. Single-page application platforms answer any unrecognised path with the app shell and a 200, so a missing file and a present one look identical to a status-code check. Verified on 9 September 2026, one live Replit app returned a byte-identical 5,712-byte HTML body for both its llms.txt and a path invented for the test. Fetch a deliberately nonexistent path first and fail the build if that returns 200.

The gate failed again immediately after I deployed the fix. What went wrong?

Results are cached for five minutes, so a scan run seconds after a deploy re-reads the pre-fix report. Add the --fresh flag to any scan step that runs later in the same pipeline as a deploy, or move the scan into a separate job that starts after the deployment has settled.

npx aiscan-cli crashes on my runner with a syntax error. What is wrong?

The CLI needs Node 18 or later and will not run below that. Add a setup-node step pinning 18 or newer, or use the dependency-free fallback the CLI documents: curl -fsSL https://aiscan.site/cli.mjs | node - example.com. If npm itself is missing from a minimal image, that same fallback is the answer.

My control path correctly returns 404, but the llms.txt I fetch still comes back as HTML. Why?

A correct 404 proves the router is honest, not that the file has the right type. Check the response content type as well as the status: a plain-text file should answer text/plain, and HTML there usually means a redirect landed you on a page. Fetch with curl -sL and print both the status code and the content type, then assert on both values.

What exit codes does the CLI return, and which one means the gate failed?

Four codes, all verified on 9 September 2026 against live sites: 0 pass, 1 gate failed, 2 bad usage, and 3 network or API error. Only 1 means your site regressed. A 2 means the command line was malformed and a 3 means the scan never completed, so treat those as pipeline faults rather than site faults if you alert on failures.

Should I fail the build on the score or on the essential checks?

Both, with different jobs in mind. The --fail-on essential flag catches things that break agent access outright, such as a missing robots.txt or no AI crawler rules. A --min-score set slightly below today's number catches slow drift. Running them together, as the workflow in this guide does, gives you a hard floor and a ratchet.

Can I gate a WordPress or Shopify site in CI when there is no build step?

Not before deploy, because there is no build folder to assert against. Use a scheduled workflow against the live URL instead, running weekly, and let a plugin or app own the files between runs. On WordPress that means one plugin holding robots.txt, robots meta, schema, sitemaps and llms.txt rather than several writing over each other; on Shopify it means an app that regenerates llms.txt as the catalogue changes.

How often should the scheduled scan run, and does the timing matter?

Weekly is enough for most sites, and daily if you deploy continuously. According to GitHub's own events reference, scheduled workflows run in UTC, run on the latest commit on the default branch, and cannot be scheduled more often than every five minutes. Pick an off-peak hour so a queued runner does not delay the result.