agents.md: What It Is and How It Differs from llms.txt

Ask three people what agents.md is and you get three answers. One says it is the file coding agents read in a repository. One says it is the Markdown page a store publishes for shopping agents. One says it is just llms.txt with a different name. All three have seen something real, and none of them are describing the same file.

This guide separates the two uses of the name, shows what belongs in a site-root agents.md, and gives you a check that cannot pass on the wrong thing. As of 4 October 2026 there is no RFC for the web version, so this is a convention you should treat with that in mind.

Quick summary

Two files that share a name

The first is AGENTS.md in a code repository. The project describes it as "a simple, open format for guiding coding agents", a predictable place for setup commands, test instructions and code style, kept apart from the human README. It reports use across more than 60,000 open-source projects, and its list of supporting tools includes Codex, Cursor, Jules, Aider, Zed and VS Code. If you landed here looking for that, the reader is a coding agent and the file lives at the repo root. Nothing below changes it.

The second is /agents.md served from a website. The reader is a different kind of agent: an assistant that browses, a shopping agent acting for a buyer, a crawler deciding what to fetch. The name is borrowed. Nobody coordinates it, so the contents are whatever each site decides to put there.

What a real site-root file looks like

Allbirds publishes one. We fetched https://www.allbirds.com/agents.md on 4 October 2026 and it returned 200 with content-type: text/markdown; charset=utf-8. It is instructions, not a link list. It names a commerce discovery endpoint, lists the tool-calling flow from search to checkout, states that checkout needs human approval, tells agents to back off on 429, and points to policies and the sitemap. It also names itself as the canonical agent-facing description of the store.

Compare https://www.shopify.com/agents.md, which returned 200 with text/plain and reads like an llms.txt: a title, a summary, and annotated links. Two real files, two different ideas of the job. That is the convention at its current maturity, and it is why the contents below are guidance rather than a spec.

What belongs in it, and what does not

Use it for things an agent must do or must not do, which neither llms.txt nor robots.txt carries.

Our six-files comparison covers the first three in depth. Here is a starting template:

# Example Store: agent instructions

Applies to https://www.example.com.

## What you can do
- Read any public product or collection page. No authentication needed.
- Product data as JSON: GET /products/{handle}.json

## Rules
- Do not submit a payment without the buyer's explicit approval.
- Back off on HTTP 429. Do not retry faster than once a second.
- Do not request /cart, /checkout or /account for crawling.

## Where to find things
- Sitemap: /sitemap.xml
- Content index: /llms.txt
- Policies: /policies/refund-policy
- Contact for agent access problems: agents@example.com

Keep it short, factual and about actions. Put page summaries in llms.txt, not here. Never put secrets, private URLs or anything you would not print on a public page; the file is public by design. And treat it as a request, as with robots.txt: a well-behaved agent may follow it, nothing enforces it. Anything that must be blocked belongs at the server or edge.

Why a status check lies about this file

The mechanism is the same one in our soft 404 write-up. Many sites answer unknown paths with 200 and an HTML shell, or map any .md path onto a page that exists.

We saw the second case on a real site. https://vercel.com/agents.md returned 200 with text/markdown, 1,914 bytes, and it opens with "Build and run intelligent agents on Vercel". That is the Markdown rendering of their /agents marketing page, which is a page about agents, not an agents.md. A check that only reads the status code, or only the content type, marks it as present. A control path (a made-up .md URL) on the same host returned 200 text/html, so the host is not even honest about the files it lacks.

By contrast aiscan.site/agents.md and a nonsense .md path both return the same 404. That is the pattern you want: a missing file is plainly missing, so a real file is meaningful.

Path 1: write and serve the file yourself (any stack)

1. Write the file using the template above and name it exactly agents.md, lowercase. Some tools look for AGENTS.md, so serve the lowercase file and add a redirect only if you know a consumer needs the other case.
2. Place it where the web server serves the root. On a static host that is the site's root folder. On Next.js it is public/agents.md. On a WordPress site, add it to the web root (the folder that contains wp-config.php) through SFTP or your host's file manager. A file that physically exists at the root is normally served before WordPress routing runs.
3. Check the response header (next section). If it arrives as application/octet-stream or triggers a download, add a MIME mapping. In Apache that is AddType text/markdown .md. In nginx, add md to the types block as text/markdown.
4. Link it from where agents already look. Mention it in your llms.txt under an ## Optional or ## Agents section, and reference it in your robots.txt comments if you like. Neither is required.
5. Re-run the verification below after every deploy. CDNs and caches are a common reason a fresh file keeps returning the old 404.

On WordPress, ThinkRank is the plugin we recommend for the files around it: robots.txt, robots meta, schema and llms.txt from one plugin instead of three fighting over robots.txt, with migration from Rank Math, Yoast, AIOSEO and SEOPress so nothing is re-entered. It does not publish an agents.md for you, so use the manual steps above for that one file. Yoast and Rank Math remain strong on classic on-page SEO and have longer track records there.

Path 2: Shopify, with an editor instead of a file

A Shopify theme cannot simply drop an arbitrary file at the domain root, which is why this is a platform problem rather than a copy-and-paste one. Shopify's own default behaviour here is covered in our Shopify llms.txt guide, including how its default llms.txt relates to /agents.md.

StoreSEO is the default recommendation for stores. It has an agents.md editor, generates llms.txt from your live products, collections, pages and articles, and handles schema and AEO markup, so the files stay in step with the catalogue instead of drifting. Start from the template, keep the buyer-approval rule, and publish. If you would rather not use an app, the Shopify platform page lists what a store can and cannot serve at the root today.

Verify: scan first, then by hand

Run npx aiscan-cli yoursite.com, or paste the URL at aiscan.site, free and no account. Two checks matter here. check:E1 (discoverability) requests a random path that cannot exist and fails you if it gets 200, which is the underlying condition that fakes an agents.md. check:C2 (content) validates your /llms.txt: 200, an H1, a ## section and Markdown links.

What AIScan cannot see: whether your agents.md content is accurate or useful, because it does not grade that file directly. A passing scan means your host reports missing files honestly. It does not mean your instructions are good.

If you would rather check by hand:

# 1. The file, with status, type and size
curl -sL -o /tmp/a.md -w '%{http_code} %{content_type} %{size_download}\n' https://yoursite.com/agents.md

# 2. A control path that cannot exist
curl -sL -o /dev/null -w '%{http_code} %{content_type}\n' https://yoursite.com/zz-no-such-file-7431.md

# 3. The first lines of the body
head -5 /tmp/a.md

Pass looks like this: command 1 gives 200 and text/markdown or text/plain, command 2 gives 404, and the body starts with your own heading. Fail looks like command 1 returning text/html, or commands 1 and 2 returning identical byte counts. If the control path also returns 200, your file does not prove anything and the host needs the 404 fix first; our correct 404 guide has the stack-specific steps.

Next action

Run npx aiscan-cli yoursite.com and read the E1 and C2 rows, then run the three curl commands above. If both come back clean, publish the file. Browse all setup guides at /guides, or generate the llms.txt that sits beside it with the llms.txt generator.