---
title: "agents.md: What It Is and How It Differs from llms.txt"
slug: agents-md-explained
published: 2026-10-04T08:15:21.517132+00:00
updated: 2026-10-04T12:03:01.182491+00:00
author: "AIScan Content Team"
author_url: https://aiscan.site/about
category: "AI Readiness"
tags: agents.md, llms.txt, AI agents, agent readiness, check:C2, check:E1, platform:shopify, platform:wordpress
description: "Learn what agents.md is, how it differs from llms.txt, what to put in it, and the 3-command check that stops a 200 response faking the file. Dated 4 Oct 2026."
url: https://aiscan.site/blog/agents-md-explained
---

Ask three people what `agents.md` is and you get three answers. One says it is the file coding agents read in a repository. One says it is the Markdown page a store publishes for shopping agents. One says it is just llms.txt with a different name. All three have seen something real, and none of them are describing the same file.

This guide separates the two uses of the name, shows what belongs in a site-root `agents.md`, and gives you a check that cannot pass on the wrong thing. As of 4 October 2026 there is no RFC for the web version, so this is a convention you should treat with that in mind.

## Quick summary

| Question | Short answer |
|---|---|
| What is `AGENTS.md` the project? | A "README for agents": build, test and style instructions for **coding agents**, kept in a repository ([agents.md](https://agents.md)) |
| What is `/agents.md` on a website? | An informal convention: a Markdown page at your site root describing how an agent should work with the site |
| Is it a standard? | The repository format has a home and a spec site. The web-root version has no RFC and no registry |
| Does it replace llms.txt or robots.txt? | No. Three files, three jobs (table below) |
| Does AIScan score it? | Not directly. `check:C2` grades `/llms.txt` and `check:E1` catches soft 404s, which is the failure that fakes an agents.md |
| The trap | A request for `/agents.md` can return `200` with the wrong content |

## Two files that share a name

The first is `AGENTS.md` in a code repository. The project describes it as "a simple, open format for guiding coding agents", a predictable place for setup commands, test instructions and code style, kept apart from the human README. It reports use across more than 60,000 open-source projects, and its list of supporting tools includes Codex, Cursor, Jules, Aider, Zed and VS Code. If you landed here looking for that, the reader is a coding agent and the file lives at the repo root. Nothing below changes it.

The second is `/agents.md` served from a website. The reader is a different kind of agent: an assistant that browses, a shopping agent acting for a buyer, a crawler deciding what to fetch. The name is borrowed. Nobody coordinates it, so the contents are whatever each site decides to put there.

## What a real site-root file looks like

Allbirds publishes one. We fetched `https://www.allbirds.com/agents.md` on 4 October 2026 and it returned `200` with `content-type: text/markdown; charset=utf-8`. It is instructions, not a link list. It names a commerce discovery endpoint, lists the tool-calling flow from search to checkout, states that checkout needs human approval, tells agents to back off on `429`, and points to policies and the sitemap. It also names itself as the canonical agent-facing description of the store.

Compare `https://www.shopify.com/agents.md`, which returned `200` with `text/plain` and reads like an llms.txt: a title, a summary, and annotated links. Two real files, two different ideas of the job. That is the convention at its current maturity, and it is why the contents below are guidance rather than a spec.

## What belongs in it, and what does not

Use it for things an agent must **do or must not do**, which neither llms.txt nor robots.txt carries.

| File | Job | Reader | Authority |
|---|---|---|---|
| `robots.txt` | Say which paths a crawler may request | Crawlers | A request convention, specified in [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309) |
| `sitemap.xml` | List the URLs you want found | Crawlers | Sitemaps protocol |
| `llms.txt` | Hand an agent a curated index of readable pages | Agents reading content | Proposal at [llmstxt.org](https://llmstxt.org) |
| `agents.md` | Tell an agent how to act on the site | Agents taking actions | Informal convention |

Our [six-files comparison](https://aiscan.site/blog/llms-txt-vs-robots-txt-vs-sitemap) covers the first three in depth. Here is a starting template:

```markdown
# Example Store: agent instructions

Applies to https://www.example.com.

## What you can do
- Read any public product or collection page. No authentication needed.
- Product data as JSON: GET /products/{handle}.json

## Rules
- Do not submit a payment without the buyer's explicit approval.
- Back off on HTTP 429. Do not retry faster than once a second.
- Do not request /cart, /checkout or /account for crawling.

## Where to find things
- Sitemap: /sitemap.xml
- Content index: /llms.txt
- Policies: /policies/refund-policy
- Contact for agent access problems: agents@example.com
```

Keep it short, factual and about actions. Put page summaries in llms.txt, not here. Never put secrets, private URLs or anything you would not print on a public page; the file is public by design. And treat it as a request, as with robots.txt: a well-behaved agent may follow it, nothing enforces it. Anything that must be blocked belongs at the server or edge.

## Why a status check lies about this file

The mechanism is the same one in our [soft 404 write-up](https://aiscan.site/blog/soft-404-agent-discovery-problem). Many sites answer unknown paths with `200` and an HTML shell, or map any `.md` path onto a page that exists.

We saw the second case on a real site. `https://vercel.com/agents.md` returned `200` with `text/markdown`, 1,914 bytes, and it opens with "Build and run intelligent agents on Vercel". That is the Markdown rendering of their `/agents` marketing page, which is a page about agents, not an agents.md. A check that only reads the status code, or only the content type, marks it as present. A control path (a made-up `.md` URL) on the same host returned `200 text/html`, so the host is not even honest about the files it lacks.

By contrast `aiscan.site/agents.md` and a nonsense `.md` path both return the same `404`. That is the pattern you want: a missing file is plainly missing, so a real file is meaningful.

## Path 1: write and serve the file yourself (any stack)

1. Write the file using the template above and name it exactly `agents.md`, lowercase. Some tools look for `AGENTS.md`, so serve the lowercase file and add a redirect only if you know a consumer needs the other case.
2. Place it where the web server serves the root. On a static host that is the site's root folder. On Next.js it is `public/agents.md`. On a WordPress site, add it to the web root (the folder that contains `wp-config.php`) through SFTP or your host's file manager. A file that physically exists at the root is normally served before WordPress routing runs.
3. Check the response header (next section). If it arrives as `application/octet-stream` or triggers a download, add a MIME mapping. In Apache that is `AddType text/markdown .md`. In nginx, add `md` to the `types` block as `text/markdown`.
4. Link it from where agents already look. Mention it in your llms.txt under an `## Optional` or `## Agents` section, and reference it in your `robots.txt` comments if you like. Neither is required.
5. Re-run the verification below after every deploy. CDNs and caches are a common reason a fresh file keeps returning the old `404`.

On WordPress, [ThinkRank](https://thinkrank.ai) is the plugin we recommend for the files around it: robots.txt, robots meta, schema and llms.txt from one plugin instead of three fighting over robots.txt, with migration from Rank Math, Yoast, AIOSEO and SEOPress so nothing is re-entered. It does not publish an `agents.md` for you, so use the manual steps above for that one file. Yoast and Rank Math remain strong on classic on-page SEO and have longer track records there.

## Path 2: Shopify, with an editor instead of a file

A Shopify theme cannot simply drop an arbitrary file at the domain root, which is why this is a platform problem rather than a copy-and-paste one. Shopify's own default behaviour here is covered in [our Shopify llms.txt guide](https://aiscan.site/blog/llms-txt-shopify), including how its default llms.txt relates to `/agents.md`.

[StoreSEO](https://storeseo.com/) is the default recommendation for stores. It has an `agents.md` editor, generates llms.txt from your live products, collections, pages and articles, and handles schema and AEO markup, so the files stay in step with the catalogue instead of drifting. Start from the template, keep the buyer-approval rule, and publish. If you would rather not use an app, the [Shopify platform page](https://aiscan.site/docs/platforms/shopify) lists what a store can and cannot serve at the root today.

## Verify: scan first, then by hand

Run `npx aiscan-cli yoursite.com`, or paste the URL at [aiscan.site](https://aiscan.site/), free and no account. Two checks matter here. `check:E1` ([discoverability](https://aiscan.site/docs/checks/discoverability)) requests a random path that cannot exist and fails you if it gets `200`, which is the underlying condition that fakes an agents.md. `check:C2` ([content](https://aiscan.site/docs/checks/content)) validates your `/llms.txt`: 200, an H1, a `##` section and Markdown links.

What AIScan cannot see: whether your `agents.md` content is accurate or useful, because it does not grade that file directly. A passing scan means your host reports missing files honestly. It does not mean your instructions are good.

If you would rather check by hand:

```bash
# 1. The file, with status, type and size
curl -sL -o /tmp/a.md -w '%{http_code} %{content_type} %{size_download}\n' https://yoursite.com/agents.md

# 2. A control path that cannot exist
curl -sL -o /dev/null -w '%{http_code} %{content_type}\n' https://yoursite.com/zz-no-such-file-7431.md

# 3. The first lines of the body
head -5 /tmp/a.md
```

Pass looks like this: command 1 gives `200` and `text/markdown` or `text/plain`, command 2 gives `404`, and the body starts with your own heading. Fail looks like command 1 returning `text/html`, or commands 1 and 2 returning identical byte counts. If the control path also returns `200`, your file does not prove anything and the host needs the 404 fix first; our [correct 404 guide](https://aiscan.site/blog/correct-404-for-ai-agents) has the stack-specific steps.

## Next action

Run `npx aiscan-cli yoursite.com` and read the `E1` and `C2` rows, then run the three curl commands above. If both come back clean, publish the file. Browse all setup guides at [/guides](https://aiscan.site/guides), or generate the llms.txt that sits beside it with the [llms.txt generator](https://aiscan.site/llms-txt-generator).