AIScan.site
Platform playbook

AI-agent readiness for custom and unrecognised stacks

This is the largest group we scan: bespoke applications, in-house frameworks, older PHP and Rails estates, and sites behind a CDN that hides whatever is underneath. Nothing here is broken by definition — the point is that every safety net a platform would have given you is absent. Whatever robots.txt, sitemap, structured data or feed exists is there because someone wrote it, and the report is usually a list of the ones nobody got to.

D1 · robots.txt present & saneD2 · XML sitemapC2 · /llms.txtC3 · Structured HTML (title, meta, JSON-LD, single H1)B2 · Explicit AI bot rulesE1 · Correct 404 handlingE4 · HTTPS & canonical host

What our own scans say about Custom / Unknown

Not an estimate. These numbers come from public AIScan reports and change as we scan more.

Custom / Unknown sites scanned
216

Distinct public domains, counted once each at their most recent scan.

Average score
44/100

Mean of the latest public scan for every Custom / Unknown domain we have graded.

What Custom / Unknown sites fail most often

Computed from public AIScan reports, refreshed as we scan. Private reports are never included.

How Custom / Unknown sites fail, and what fixes it

Nobody owns the root of the domain

robots.txt, sitemap.xml, llms.txt and .well-known are shared infrastructure with no obvious team. On a custom stack they are the first things to go missing and the cheapest to restore.

D1 · robots.txt present & saneD2 · XML sitemapC2 · /llms.txt

Canonical host drift

www and apex both answering 200, or http not redirecting, splits every signal you have in half and makes an agent unsure which of the two it is quoting. Check E4 grades that directly.

E4 · HTTPS & canonical host

Soft 404s from the application layer

Frameworks that render a friendly 'not found' page with a 200 status teach a crawler that every guessed URL exists, which is worse than a hard failure.

E1 · Correct 404 handling

Structured data that was never added

Without a CMS emitting it, JSON-LD tends not to exist at all. One Organization block plus the type that matches your business is a short change with a disproportionate effect on how accurately you get summarised.

C3 · Structured HTML (title, meta, JSON-LD, single H1)

Step-by-step fixes: /docs · /docs/checks/discoverability · /docs/checks/content · /llms-txt-generator

Businesses running on Custom / Unknown

AI readiness for event venues and organisers

Capacity, dates and hire fee — the three things every enquiry asks and the page never says.

Questions Custom / Unknown owners ask

Our platform was not detected. Does that lower the score?
No. Detection only chooses which fix guide you are shown. Scoring is identical for every stack.
Which fix should a custom site do first?
robots.txt with an explicit AI position, then a complete sitemap, then llms.txt. Those three are the shortest path from invisible to legible.
How do we keep this from regressing?
Put the CLI in CI: npx aiscan-cli yoursite.com --min-score 85 --fail-on essential fails the build when someone removes a header or breaks the sitemap.
Is a CDN in front of us a problem?
Only when it caches an old robots.txt or strips headers. If a fix does not register, purge and re-scan with fresh=1.
We have an internal API. Should it be discoverable?
Public endpoints, yes — describe them and link the description from an API catalogue. Internal ones should stay behind auth and out of the catalogue entirely.

Scan your Custom / Unknown site and see where you actually stand

Free, no account, about twenty seconds. The report names every failing check by ID, shows the evidence we found, and gives the fix for your platform — plus a hand-off prompt you can paste straight into Claude Code or Cursor.

Run a free scan npx aiscan-cli yoursite.com

Prefer to read first? Browse every check we run or the guide library.