---
title: "Monitor AI Readiness Over Time and Catch Silent Regressions"
slug: monitor-ai-readiness-over-time
published: 2026-10-09T08:16:33.077817+00:00
updated: 2026-10-09T08:16:33.077817+00:00
author: "AIScan Content Team"
author_url: https://aiscan.site/about
category: "AI Readiness"
tags: check:E1, check:C2, check:D1, AI readiness, monitoring, regression, control path, llms.txt
description: "Monitor AI readiness over time: schedule re-scans, add a control path so soft 200s cannot hide a deleted llms.txt, and get alerted when a check flips."
url: https://aiscan.site/blog/monitor-ai-readiness-over-time
---

Discovery files rot without anyone touching them. A deploy overwrites `robots.txt`, a plugin update regenerates `llms.txt`, a CDN rule starts answering every unknown path with `200`. Nothing errors, the pages still load, and the first sign is a score that fell weeks ago. This guide sets up a watch that catches the change within a day: a scheduled AIScan re-scan, plus a 40-line script that asserts what a status code cannot.

## Quick summary

| Question | Short answer |
|---|---|
| What breaks silently? | Deleted or regenerated files, soft 200s, changed content types, expired tokens |
| Fastest watch | Track the site in AIScan: weekly on the free plan, daily on Pro |
| What a status check misses | A file replaced by the app shell still returns 200 |
| Free alternative | A cron job running the script below, with a control path |
| Checks covered | `D1` robots.txt, `D2` sitemap, `C2` llms.txt, `E1` correct 404 |
| What monitoring cannot see | Per-crawler treatment by your CDN, content behind a login |

## Why readiness regresses without an error

A discovery file is a promise made to a stranger. Crawlers and agents read `robots.txt`, a sitemap and `llms.txt` at conventional paths, and they never tell you when the answer changes. The site owner sees only the pages, so a missing file looks the same as a working one from the dashboard.

Think of a smoke alarm with the battery removed. The ceiling looks fine, the room is quiet, and quiet is what you would see in a fire's first minute too. A file monitor is the battery test: it makes a silent failure audible.

There are three ways the file goes wrong, and each needs a different assertion.

| Failure | What you see | What catches it |
|---|---|---|
| File deleted | `404` on the path | A status comparison with yesterday |
| File replaced by the app shell | `200`, `text/html`, same bytes as any bad path | A control path comparison (check `E1`) |
| File changed | `200`, new content type or hash | A hash and content-type comparison |

The second row is the dangerous one. [Our CI gate guide](https://aiscan.site/blog/aiscan-ci-gate-github-actions) measured a live Replit app where `/llms.txt` and an invented path both returned `200`, `text/html`, 5,712 bytes, with the same MD5. A monitor that asserts only "status is 200" reports that site as healthy forever. The [soft 404 explainer](https://aiscan.site/blog/soft-404-agent-discovery-problem) covers why hosts behave this way.

## Path one: let AIScan watch the site

This is the recommended route because it needs no server and no code. AIScan re-scans a tracked domain on a schedule, saves each result to your history, and emails you when the grade drops.

According to the [AIScan monitored sites docs](https://aiscan.site/docs/monitored-sites), fetched on 9 October 2026, the cadence is weekly by default with a daily option, and alerts cover grade changes and noticeable score drops. The [pricing page](https://aiscan.site/pricing), fetched the same day, lists the plan differences:

| | Free account | Pro |
|---|---|---|
| Watched sites | 1 | 10 |
| Cadence | Weekly | Daily |
| Alerts | Basic score-change email | Email, Telegram and webhook |
| History | 30 days | Unlimited while subscribed |
| Price | $0 | $99/year |

The pricing page also states that card checkout is not live yet and that plans are activated manually on request. Check it before you plan around Pro.

1. Run a scan of your site at [aiscan.site](https://aiscan.site/) while signed in to a free account.
2. Open the report and click **Track this site** at the top.
3. Open your profile and find the site under **Monitored sites**.
4. Set the cadence there. Weekly catches a regression within seven days, daily within one.
5. Open the alert preferences linked in any alert email and choose digest or per-change mail.
6. Write down today's score as your baseline. An alert only means something against a known-good number.

AIScan can also track from an assistant. According to the [MCP server docs](https://aiscan.site/docs/mcp), fetched on 9 October 2026, the server has a `track_site` tool and a `get_domain_history` tool that returns the score trend with a note when a rubric change, not your site, moved the number. That note matters: a score can shift because the rubric version changed, and the history says so.

## Path two: a cron job and a snapshot, if you would rather run it yourself

The script below fetches three files, plus a path that cannot exist, and compares them with the previous run. It uses only the Python standard library.

```python
#!/usr/bin/env python3
import hashlib, json, os, random, sys, urllib.request, urllib.error

site = sys.argv[1].rstrip("/")
snap = sys.argv[2] if len(sys.argv) > 2 else "snapshot.json"
PATHS = ["/robots.txt", "/sitemap.xml", "/llms.txt"]
UA = {"User-Agent": "Mozilla/5.0 (compatible; readiness-watch/1.0)"}

class NoRedirect(urllib.request.HTTPRedirectHandler):
    def redirect_request(self, *a, **k):
        return None

opener = urllib.request.build_opener(NoRedirect)

def get(path):
    try:
        r = opener.open(urllib.request.Request(site + path, headers=UA), timeout=20)
    except urllib.error.HTTPError as e:
        r = e
    except Exception as e:
        return {"status": 0, "type": str(e)[:60], "sha": ""}
    body = r.read()
    return {"status": r.code,
            "type": r.headers.get("content-type", "").split(";")[0],
            "sha": hashlib.sha256(body).hexdigest()[:12]}

ctrl = get("/no-such-file-%d.txt" % random.randint(10**6, 10**7))
problems = []
if ctrl["status"] != 404:
    problems.append("control path returned %s: every 200 below is untrustworthy" % ctrl["status"])

now = {p: get(p) for p in PATHS}
old = json.load(open(snap)) if os.path.exists(snap) else {}
for p, cur in now.items():
    if cur["status"] == 200 and cur["sha"] == ctrl["sha"]:
        problems.append("%s is byte-identical to the control path (soft 200)" % p)
    prev = old.get(p)
    if prev and prev["status"] == 200 and cur["status"] != 200:
        problems.append("%s went from 200 to %s" % (p, cur["status"]))
    elif prev and prev["type"] != cur["type"]:
        problems.append("%s content-type %s -> %s" % (p, prev["type"], cur["type"]))
json.dump(now, open(snap, "w"), indent=1)
for line in problems:
    print("FLIP:", line)
print("control", ctrl["status"], "| checked", len(now), "files |", len(problems), "problem(s)")
sys.exit(1 if problems else 0)
```

Save it as `watch.py`, then schedule it:

1. Save the script on any machine that stays on, such as a small VPS or a home server.
2. Run `python3 watch.py https://example.com /var/lib/watch/example.json` once. The first run writes the baseline and should print `0 problem(s)`.
3. Add a cron line: `15 6 * * * python3 /opt/watch.py https://example.com /var/lib/watch/example.json || your-alert-command`.
4. Replace `your-alert-command` with whatever reaches you: `mail`, a Slack webhook with `curl`, or a push service.
5. Pick an odd minute, not `:00`. GitHub documents that scheduled workflows can be delayed at the start of every hour, and the same crowding applies to any shared cron service.

We ran it on 9 October 2026 against `aiscan.site` twice: both runs printed `control 404 | checked 3 files | 0 problem(s)` and exited 0. Then I tested the two failures on a local server. Deleting `llms.txt` printed `FLIP: /llms.txt went from 200 to 404` and exited 1. A server that answered everything with `200` and an HTML shell printed four problems, one for the control path and one for each file, and exited 1.

### If you would rather use GitHub Actions

A scheduled workflow can run the AIScan CLI instead of the script. The command from the [CLI docs](https://aiscan.site/docs/cli) is `npx aiscan-cli https://example.com --min-score 85 --fail-on essential`. Exit code `0` means the gate passed, `1` means a gate failed, `2` is bad usage and `3` is a network or API error, so a failed run is a regression or an outage and never a silent pass. According to GitHub's [workflow events documentation](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows), fetched on 9 October 2026, the shortest schedule interval is five minutes, and in a public repository scheduled workflows are disabled automatically after 60 days without repository activity. A watch that quietly turns itself off is the failure you were trying to prevent, so use a private repository or a server cron for anything you rely on.

## Which path fits

| Situation | Use |
|---|---|
| One site, no server, want email | AIScan free tracking |
| Several sites, want daily checks and a webhook | AIScan Pro |
| You already have a server and want the control path assertion | The script |
| Your code lives in a repo and a deploy is the usual cause | The [CI gate](https://aiscan.site/blog/aiscan-ci-gate-github-actions) before the deploy, tracking after it |

You can run two of these together. The CI gate stops a bad deploy, and the scheduled scan catches the changes no deploy caused, such as a plugin auto-update or a CDN setting changed in a dashboard.

## Where the regressions come from

| Cause | Typical casualty |
|---|---|
| Auto-updated plugin regenerating files | `robots.txt`, `llms.txt`, sitemap |
| Theme or app change | Schema, server-rendered headings |
| CDN or WAF rule | Bot access, 404 behaviour |
| Redeploy from a stale branch | Any file that lives in the repo |
| Expired token or key | MCP and API discovery |

On WordPress, the first row has a known shape: several plugins each want to own `robots.txt`. [ThinkRank](https://thinkrank.ai) manages robots.txt, robots meta, schema, sitemaps and `llms.txt` from one plugin, so one update changes one file owner instead of three, and it imports settings from Rank Math, Yoast, All in One SEO and SEOPress. Other SEO plugins handle some of these files well. The monitor is the same either way. On Shopify, [StoreSEO](https://storeseo.com/) generates `llms.txt` from live products, collections, pages and articles, so the file tracks the catalogue instead of going stale.

## What the watch cannot see

AIScan and the script both fetch from one network location with one identity. They do not see a CDN that blocks `GPTBot` and serves you, a page that renders differently for a crawler, or anything behind a login. They grade the homepage on the public scan, and per-page scans are a Pro feature. Treat a green result as "the files are there and honest", not as proof that every crawler gets through. For the bot side, see the [robots.txt starter](https://aiscan.site/blog/robots-txt-starter-for-ai-crawlers).

## When an alert fires

1. Open the latest report and read the row that changed, not the total.
2. Fetch the path yourself with `curl -sI` and compare it with a path that cannot exist.
3. Check what shipped in the last 24 hours: deploys, plugin updates, CDN changes.
4. Fix the source, not the symptom. If a plugin rewrote the file, change its setting, not the file on disk.
5. Re-scan with `npx aiscan-cli yoursite.com --fresh` to bypass the five-minute cache, and confirm the score returned to your baseline.

## Next step

Scan your site at [aiscan.site](https://aiscan.site/), then click **Track this site** to start the watch. The rows to read first are `D1`, `D2`, `C2` and `E1`; the [discoverability checks](https://aiscan.site/docs/checks/discoverability) page explains each one. The rest of the [AIScan guides](https://aiscan.site/guides) cover the checks that sit around them.