AIScan.site
Content Checks

Content Checks

C1 Markdown content negotiation, C2 llms.txt, C3 structured HTML.

Agents don't render HTML the way a browser does. They strip it down, look for headings and meta, and try to extract clean prose. Help them by exposing content in formats they don't have to fight. None of this matters if crawlers can't find or aren't allowed to reach your pages — pair these with the Discoverability checks and Bot Access checks.

C1 — Markdown content negotiation (weight 8)

AIScan re-requests your homepage with Accept: text/markdown. If the server replies with atext/markdown content-type and the body looks like markdown (headings, lists, links), the check passes.

How to fix

Two approaches:

  • Content negotiation — at the edge or in your framework, when Accept includes text/markdown, render the same page as Markdown instead of HTML.
  • Sidecar URLs — also serve /path/index.md alongside /path. Easier for static sites.

C2 — /llms.txt (weight 6)

Per llmstxt.org, /llms.txt is a markdown file that gives agents a curated index of your site. AIScan checks that the file exists, returns 200, and contains an H1, at least one ## section, and markdown links.

How to fix

Drop a file at the root of your site that looks like the example below — or generate one with the llms.txt generator:

# Acme Inc.

> A short summary of what Acme does — one or two sentences.

## Docs
- [Getting started](https://acme.com/docs/getting-started.md)
- [API reference](https://acme.com/docs/api.md)

## Examples
- [Example app](https://acme.com/examples/app.md)

C3 — Structured HTML (weight 4)

Four basic signals on your homepage: a single <h1>, a non-empty <title>, a <meta name="description">, and at least one application/ld+jsonblock. One point each.

How to fix

For JSON-LD, start with an Organization or WebSite schema sitewide, then add page-type-specific schemas (Article, Product, FAQPage) on the pages that match. Validators: schema.org validator.

E3 — Heading hierarchy & server-rendered text (weight 4, essential)

AIScan strips scripts and styles from the HTML your server actually returns, then counts the remaining words and the <h2>/<h3> outline. This is what a non-JavaScript crawler receives — most LLM crawlers do not execute your bundle.

  • 150+ words of server-rendered text: 2 points (50+: 1 point).
  • Two or more <h2> sections: 2 points (one: 1 point).

How to fix

Server-render or pre-render your marketing and docs copy, and structure it with one <h1> then descriptive <h2>/<h3> sections. Check what a crawler sees:

curl -s https://yoursite.com/ | sed 's/<[^>]*>/ /g' | tr -s ' ' | head -c 800

E5 — Content feed (RSS / Atom / JSON Feed) (weight 3, recommended)

A feed is a dated changelog of your site — a freshness signal a sitemap doesn't carry. AIScan looks for a feed at the conventional paths and checks whether it's declared in <head> with <link rel="alternate">. Declared feed: full credit. Found by convention only: partial.

This check is graded only where periodical content is expected (a blog, news, or articles section, or a platform like WordPress or Ghost). Otherwise it reports N/A with the reason, and costs nothing.

How to fix

<link rel="alternate" type="application/rss+xml" title="Blog" href="/feed.xml" />

Further reading