01Fail
Markdown content negotiation
+8 pts if fixed
Serve text/markdown when the client sends Accept: text/markdown, or publish /index.md.
See full details
02Fail
/llms.txt
+6 pts if fixed
Publish /llms.txt per llmstxt.org with an H1, summary, and ## sections of markdown links.
See full details
03Fail
Explicit AI bot rules
+5 pts if fixed
Add explicit allow/deny rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.
See full details

Hand-off prompt for your coding agent9 issues

# Agent-readiness remediation brief

Site: https://www.iana.org/
Platform: unknown (0% confidence)
AIScan score: 33/100 — Level 2 (Readable)
Rubric: v2026.08.2

## Failing checks (sorted by weight)
- **[C1] Markdown content negotiation** — FAIL
  - Evidence: Accept: text/markdown → 200 (text/html; charset=UTF-8)
  - Fix: Serve text/markdown when the client sends Accept: text/markdown, or publish /index.md.
- **[D1] robots.txt present & sane** — PARTIAL
  - Evidence: 200 OK, 24 bytes
  - Fix: Serve a non-blocking /robots.txt that references your Sitemap.
- **[C2] /llms.txt** — FAIL
  - Evidence: HTTP 404
  - Fix: Publish /llms.txt per llmstxt.org with an H1, summary, and ## sections of markdown links.
- **[B2] Explicit AI bot rules** — FAIL
  - Evidence: 0 known AI agent(s) addressed in robots.txt
  - Fix: Add explicit allow/deny rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.
- **[D2] XML sitemap** — FAIL
  - Evidence: no /sitemap.xml or /sitemap_index.xml resolved
  - Fix: Publish /sitemap.xml and reference it in robots.txt.
- **[C3] Structured HTML (title, meta, JSON-LD, single H1)** — PARTIAL
  - Evidence: h1: 1, title: true, meta description: false, JSON-LD: false
  - Fix: Ensure a single <h1>, <title>, meta description, and schema.org JSON-LD.
- **[E4] HTTPS & canonical host** — PARTIAL
  - Evidence: final origin https://www.iana.org · http:// does NOT upgrade to https · iana.org redirects to the canonical host · HSTS set
  - Fix: Serve the site over HTTPS, 301-redirect http:// to https://, and redirect the www/apex variant to a single canonical host so agents don't index two copies of every URL.
- **[D3] Link header for discovery** — FAIL
  - Evidence: no Link header or <link> tag
  - Fix: Add a Link response header or HTML <link rel="api-catalog"|"describedby"> tag.
- **[E5] Content feed (RSS / Atom / JSON Feed)** — FAIL
  - Evidence: no RSS, Atom or JSON Feed found at the conventional paths
  - Fix: Publish an RSS, Atom, or JSON Feed of new content and declare it in <head> with <link rel="alternate" type="application/rss+xml">. Feeds give agents a cheap, dated changelog of your site that a sitemap doesn't.

## General guidance
- Static-first: `robots.txt`, `llms.txt`, `/.well-known/*` JSON cards.
- Markdown negotiation: serve `text/markdown` when requested via `Accept` header or `?format=md`.
- Bot access: explicit allow/deny per AI user-agent (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, Bytespider, Amazonbot, CCBot, meta-externalagent, cohere-ai, Diffbot).
- Capabilities: publish an MCP server card and (optionally) an Agent Skills manifest even if the runtime isn't live — discovery first.
- Re-scan at aiscan.site after deploying.

How this score is calculated

Every check earns points against a weight. We divide the points you earned by the points that actually apply to a site like yours — so checks that don't apply never drag you down.

Essential

17 / 31 pts

Baseline every site should meet.

Recommended

0 / 20 pts

Strong signals most sites benefit from.

Bonus earned

0 signals

Emerging standards. Can only add points.

Not counted (7): B1, B3, P1, P2, P3, P4, E2 — either not applicable to this site profile or optional and not yet adopted. Expand any check to see the exact evidence we based it on.

Discoverability

Can agents find your pages? robots, sitemaps, llms.txt.

122

Your site is missing this right now, so AI agents can't use it. The fix below is what to change.
ID · D2
What we found
no /sitemap.xml or /sitemap_index.xml resolved
Measured against
Sitemaps XML protocol 0.9 RFC 9309 §2.2.3 (Sitemap directive)
How to fix it
Publish /sitemap.xml and reference it in robots.txt.
⚡ Fix automatically with AIScan
Option A — Agent Skill (any agent runtime)
Install the AIScan skill in one command:
```
npx add-skill https://aiscan.site/aiscan-skill.json
```
Or download it directly: https://aiscan.site/aiscan-skill.json
Then ask your agent: "Fix the failing AI readiness checks for <your-url>"
Option B — Claude Code skill (CLAUDE.md)
Download CLAUDE.md from https://aiscan.site/CLAUDE.md and place it in your project root.
Ask Claude Code: "Fix the failing AIScan checks for <your-url>" — it reads the skill, calls our API, and applies the fix.
Option C — MCP (Claude Desktop, Cursor, or any MCP client)
Endpoint: https://aiscan.site/api/mcp
Tools: scan_website, get_fixes, get_grade
Call get_fixes with your URL for machine-readable fix instructions.
Option D — REST API
```
GET https://aiscan.site/api/public/scan?url=YOUR_URL
```
Returns the full report with all checks and fix details.
Re-scan at https://aiscan.site to verify after applying fixes.
Generate a sitemap
Most frameworks have a built-in sitemap generator (Next.js: app/sitemap.ts, TanStack: a route at /sitemap.xml). Publish all public URLs.
Reference it in robots.txt
Sitemap: https://yoursite.com/sitemap.xml
Verify
After deploying, re-scan at https://aiscan.site. You can also test with curl -I or curl directly.
Fix in ChatGPT Fix in Claude
Your site is missing this right now, so AI agents can't use it. The fix below is what to change.
ID · D3
What we found
no Link header or <link> tag
Measured against
RFC 8288 — Web Linking RFC 9727 — API catalog link relation
How to fix it
Add a Link response header or HTML <link rel="api-catalog"|"describedby"> tag.
⚡ Fix automatically with AIScan
Option A — Agent Skill (any agent runtime)
Install the AIScan skill in one command:
```
npx add-skill https://aiscan.site/aiscan-skill.json
```
Or download it directly: https://aiscan.site/aiscan-skill.json
Then ask your agent: "Fix the failing AI readiness checks for <your-url>"
Option B — Claude Code skill (CLAUDE.md)
Download CLAUDE.md from https://aiscan.site/CLAUDE.md and place it in your project root.
Ask Claude Code: "Fix the failing AIScan checks for <your-url>" — it reads the skill, calls our API, and applies the fix.
Option C — MCP (Claude Desktop, Cursor, or any MCP client)
Endpoint: https://aiscan.site/api/mcp
Tools: scan_website, get_fixes, get_grade
Call get_fixes with your URL for machine-readable fix instructions.
Option D — REST API
```
GET https://aiscan.site/api/public/scan?url=YOUR_URL
```
Returns the full report with all checks and fix details.
Re-scan at https://aiscan.site to verify after applying fixes.
Add a Link response header
Set this header on your homepage and/or root document. Most CDNs (Cloudflare, Vercel, Netlify) support response-header rules.
Example
Link: </.well-known/api-catalog>; rel="api-catalog"
Verify
After deploying, re-scan at https://aiscan.site. You can also test with curl -I or curl directly.
Fix in ChatGPT Fix in Claude

Content

Can agents read your content cleanly? Markdown, structured data.

113

Your site is missing this right now, so AI agents can't use it. The fix below is what to change.
ID · C1
What we found
Accept: text/markdown → 200 (text/html; charset=UTF-8)
Measured against
RFC 9110 §12 — Proactive content negotiation llmstxt.org — markdown representations
How to fix it
Serve text/markdown when the client sends Accept: text/markdown, or publish /index.md.
⚡ Fix automatically with AIScan
Option A — Agent Skill (any agent runtime)
Install the AIScan skill in one command:
```
npx add-skill https://aiscan.site/aiscan-skill.json
```
Or download it directly: https://aiscan.site/aiscan-skill.json
Then ask your agent: "Fix the failing AI readiness checks for <your-url>"
Option B — Claude Code skill (CLAUDE.md)
Download CLAUDE.md from https://aiscan.site/CLAUDE.md and place it in your project root.
Ask Claude Code: "Fix the failing AIScan checks for <your-url>" — it reads the skill, calls our API, and applies the fix.
Option C — MCP (Claude Desktop, Cursor, or any MCP client)
Endpoint: https://aiscan.site/api/mcp
Tools: scan_website, get_fixes, get_grade
Call get_fixes with your URL for machine-readable fix instructions.
Option D — REST API
```
GET https://aiscan.site/api/public/scan?url=YOUR_URL
```
Returns the full report with all checks and fix details.
Re-scan at https://aiscan.site to verify after applying fixes.
Serve text/markdown on Accept negotiation
When a client sends Accept: text/markdown, return a markdown rendering of the same content. The simplest path is to publish parallel .md files (e.g. /about → /about.md) and rewrite based on the Accept header.
Test it
curl -H "Accept: text/markdown" https://yoursite.com/
Verify
After deploying, re-scan at https://aiscan.site. You can also test with curl -I or curl directly.
Fix in ChatGPT Fix in Claude
Your site is missing this right now, so AI agents can't use it. The fix below is what to change.
ID · C2
What we found
HTTP 404
Measured against
llms.txt proposal
How to fix it
Publish /llms.txt per llmstxt.org with an H1, summary, and ## sections of markdown links.
⚡ Fix automatically with AIScan
Option A — Agent Skill (any agent runtime)
Install the AIScan skill in one command:
```
npx add-skill https://aiscan.site/aiscan-skill.json
```
Or download it directly: https://aiscan.site/aiscan-skill.json
Then ask your agent: "Fix the failing AI readiness checks for <your-url>"
Option B — Claude Code skill (CLAUDE.md)
Download CLAUDE.md from https://aiscan.site/CLAUDE.md and place it in your project root.
Ask Claude Code: "Fix the failing AIScan checks for <your-url>" — it reads the skill, calls our API, and applies the fix.
Option C — MCP (Claude Desktop, Cursor, or any MCP client)
Endpoint: https://aiscan.site/api/mcp
Tools: scan_website, get_fixes, get_grade
Call get_fixes with your URL for machine-readable fix instructions.
Option D — REST API
```
GET https://aiscan.site/api/public/scan?url=YOUR_URL
```
Returns the full report with all checks and fix details.
Re-scan at https://aiscan.site to verify after applying fixes.
Create /llms.txt at the site root
Follows the llmstxt.org spec: a single H1, optional blockquote summary, then ## sections of markdown links.
Template
# Example Co > One-line summary of what this site / product does. A short paragraph explaining the audience, what's here, and how to navigate it. ## Docs - [Getting started](/docs/quickstart): install, first call in 5 min - [API reference](/docs/api): all endpoints, params, examples ## Pages - [Pricing](/pricing): plans and limits - [Changelog](/changelog): recent releases ## Optional - [Brand assets](/press): logos and screenshots
Verify
After deploying, re-scan at https://aiscan.site. You can also test with curl -I or curl directly.
Fix in ChatGPT Fix in Claude
Your site is missing this right now, so AI agents can't use it. The fix below is what to change.
ID · E5
What we found
no RSS, Atom or JSON Feed found at the conventional paths
Measured against
RFC 4287 — Atom Syndication Format JSON Feed 1.1
How to fix it
Publish an RSS, Atom, or JSON Feed of new content and declare it in <head> with <link rel="alternate" type="application/rss+xml">. Feeds give agents a cheap, dated changelog of your site that a sitemap doesn't.
⚡ Fix automatically with AIScan
Option A — Agent Skill (any agent runtime)
Install the AIScan skill in one command:
```
npx add-skill https://aiscan.site/aiscan-skill.json
```
Or download it directly: https://aiscan.site/aiscan-skill.json
Then ask your agent: "Fix the failing AI readiness checks for <your-url>"
Option B — Claude Code skill (CLAUDE.md)
Download CLAUDE.md from https://aiscan.site/CLAUDE.md and place it in your project root.
Ask Claude Code: "Fix the failing AIScan checks for <your-url>" — it reads the skill, calls our API, and applies the fix.
Option C — MCP (Claude Desktop, Cursor, or any MCP client)
Endpoint: https://aiscan.site/api/mcp
Tools: scan_website, get_fixes, get_grade
Call get_fixes with your URL for machine-readable fix instructions.
Option D — REST API
```
GET https://aiscan.site/api/public/scan?url=YOUR_URL
```
Returns the full report with all checks and fix details.
Re-scan at https://aiscan.site to verify after applying fixes.
Publish a feed
RSS, Atom, or JSON Feed — any of them gives agents a cheap, dated changelog of new content that a sitemap can't.
Declare it in <head>
<link rel="alternate" type="application/rss+xml" title="Blog" href="/feed.xml" />
Keep dates accurate
Each item needs a real publish date; that's the freshness signal agents actually use.
Verify
After deploying, re-scan at https://aiscan.site. You can also test with curl -I or curl directly.
Fix in ChatGPT Fix in Claude

Bot Access

Are AI crawlers explicitly allowed or blocked?

Your site is missing this right now, so AI agents can't use it. The fix below is what to change.
ID · B2
What we found
0 known AI agent(s) addressed in robots.txt
Measured against
RFC 9309 — Robots Exclusion Protocol Google-Extended documentation
How to fix it
Add explicit allow/deny rules for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, etc.
⚡ Fix automatically with AIScan
Option A — Agent Skill (any agent runtime)
Install the AIScan skill in one command:
```
npx add-skill https://aiscan.site/aiscan-skill.json
```
Or download it directly: https://aiscan.site/aiscan-skill.json
Then ask your agent: "Fix the failing AI readiness checks for <your-url>"
Option B — Claude Code skill (CLAUDE.md)
Download CLAUDE.md from https://aiscan.site/CLAUDE.md and place it in your project root.
Ask Claude Code: "Fix the failing AIScan checks for <your-url>" — it reads the skill, calls our API, and applies the fix.
Option C — MCP (Claude Desktop, Cursor, or any MCP client)
Endpoint: https://aiscan.site/api/mcp
Tools: scan_website, get_fixes, get_grade
Call get_fixes with your URL for machine-readable fix instructions.
Option D — REST API
```
GET https://aiscan.site/api/public/scan?url=YOUR_URL
```
Returns the full report with all checks and fix details.
Re-scan at https://aiscan.site to verify after applying fixes.
Add explicit User-agent blocks
Don't rely on User-agent: * alone. Address each major AI crawler so your policy is unambiguous.
Recommended block
User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot-Extended Allow: /
Verify
After deploying, re-scan at https://aiscan.site. You can also test with curl -I or curl directly.
Fix in ChatGPT Fix in Claude

Showcase your grade

Embed your AIScan badge

Drop this badge into your site footer, GitHub README, or docs. It links back to a fresh scan of www.iana.org on AIScan.site.

HTML

<!-- AIScan.site agent-readiness badge -->
<a href="https://aiscan.site/?u=https%3A%2F%2Fwww.iana.org%2F" target="_blank" rel="noopener" title="Agent-Readiness Grade F (33/100) — AIScan.site">
  <img src="https://aiscan.site/api/public/badge.svg?score=33&grade=F&host=www.iana.org" alt="AIScan.site Grade F — 33/100" width="240" height="64" loading="lazy" />
</a>

Markdown

[![AIScan.site Grade F — 33/100](https://aiscan.site/api/public/badge.svg?score=33&grade=F&host=www.iana.org)](https://aiscan.site/?u=https%3A%2F%2Fwww.iana.org%2F)

Create a free account to track this site over time

Save scan history
See score trends
Re-scan anytime

Automate your AI-readiness

Plug AIScan into your agent stack or chat with it on the go.

Use the MCP server

One line in Claude Code or Cursor to scan any site from your agent.

Chat with the Telegram bot

Scan sites and get fixes right from @AIScanBot.

Free · No signup · ~20 seconds

Scan your own site

Get a plain-English agent-readiness report you can hand straight to ChatGPT or Claude.

Re-test this site Scan your site

Or browse the community feed to see what others have scanned.

iana.org

Quick Wins

Markdown content negotiation

/llms.txt

Explicit AI bot rules

How this score is calculated

robots.txt present & sane

XML sitemap

⚡ Fix automatically with AIScan

Generate a sitemap

Reference it in robots.txt

Verify

Link header for discovery

⚡ Fix automatically with AIScan

Add a Link response header

Example

Verify

Correct 404 handling

HTTPS & canonical host

Markdown content negotiation

⚡ Fix automatically with AIScan

Serve text/markdown on Accept negotiation

Test it

Verify

/llms.txt

⚡ Fix automatically with AIScan

Create /llms.txt at the site root

Template

Verify

Structured HTML (title, meta, JSON-LD, single H1)

Heading hierarchy & server-rendered text

Content feed (RSS / Atom / JSON Feed)

⚡ Fix automatically with AIScan

Publish a feed

Declare it in <head>

Keep dates accurate

Verify

Content Signals in robots.txt

Explicit AI bot rules

⚡ Fix automatically with AIScan

Add explicit User-agent blocks

Recommended block

Verify

Web Bot Auth key directory

Share as a social image

Embed your AIScan badge

Create a free account to track this site over time

Automate your AI-readiness

Scan your own site