Table of contents
- Quick summary
- Why the file lands at a URL agents will not ask for
- Route 1: the static folder
- Route 2: a postBuild hook that reads your real routes
- Route 3: an npm plugin someone already wrote
- Choosing which pages to list
- Confirm which URL you actually shipped
- Where AIScan fits, and where it doesn't
- Ship it, then re-scan
A Docusaurus site can publish a completely valid llms.txt and still score zero on it. The reason is one config value. On GitHub Pages, the platform's most common deployment target, a project site lives at https://your-org.github.io/your-project/, so baseUrl is /your-project/ and the file you dropped into static/ answers at /your-project/llms.txt. Nothing answers at the origin root. The build was fine. The URL was wrong. Below are three routes to the file, the check that tells you which URL you actually shipped, and a fix that takes about ten minutes.
Quick summary
| If you want to… | Do this | Time | Stays current on its own |
|---|---|---|---|
| Ship a file today | Put llms.txt in static/ | 5 min | No, you edit it by hand |
| Generate it from your real routes | Add an inline plugin with a postBuild hook | 20 min | Yes, every build |
| Skip writing code | Install a published plugin from npm | 10 min | Yes, every build |
| Serve it at the origin root | Deploy at baseUrl: '/', or add a host-level rewrite | Varies | N/A |
The check this maps to is C2, in AIScan's content dimension. C2 asks for a file at /llms.txt and reads whether it parses.
Why the file lands at a URL agents will not ask for
Docusaurus copies static/ straight into the build output. Its own Static Assets page says, verbatim: "Every file you put into that directory will be copied into the root of the generated build folder with the directory hierarchy preserved." The next paragraph is the one that matters here: "for site baseUrl: '/subpath/', the image /static/img/docusaurus.png will be served at /subpath/img/docusaurus.png."
That applies to every file in static/, llms.txt included. And according to Docusaurus's deployment guide, the value you need is unambiguous: "For a site deployed at https://my-org.com/my-project/, baseUrl is /my-project/." Docs sites hosted under a repository path inherit that prefix automatically.
| Deployment | baseUrl | static/llms.txt is served at | Origin root /llms.txt |
|---|---|---|---|
docs.example.com (own domain) | / | https://docs.example.com/llms.txt | 200 |
example.com/docs/ (subpath) | /docs/ | https://example.com/docs/llms.txt | 404 |
org.github.io/project/ (GitHub Pages project site) | /project/ | https://org.github.io/project/llms.txt | 404 |
A subpath file is not invalid. The llms.txt specification, fetched from llmstxt.org on 4 September 2026, allows it in as many words: "The llms.txt file spec is for files named llms.txt, at the root path /llms.txt of a website or at any subpath (e.g. /docs/llms.txt). A file covers the URLs under its path, and where more than one file applies, agents should use the most specific one."
So /docs/llms.txt is spec-legal and useful. It is also invisible to any scanner that probes only the origin root, ours included. Know which one you published before you go looking for the score.
Route 1: the static folder
Fastest path, and the right one for a hand-curated index.
- Create
static/llms.txtat the root of your Docusaurus project, besidedocusaurus.config.js. - Write the file (see the section on choosing pages below).
- Run
npm run build, thennpm run serve, and openhttp://localhost:3000plus yourbaseUrlandllms.txt. - Deploy.
If your team keeps generated assets somewhere other than static/, the staticDirectories option in docusaurus.config.js takes an array of paths, all copied to the build output as-is.
Route 2: a postBuild hook that reads your real routes
Hand-written indexes drift the moment somebody adds a page. Docusaurus hands you the finished route list at build time. According to its lifecycle API reference, postBuild(props) is "Called when a (production) build finishes" and receives siteDir, outDir, baseUrl, siteConfig, routesPaths, and routesBuildMetadata, which carries a noIndex flag per location.
Add this to docusaurus.config.js as an inline plugin:
plugins: [
function llmsTxtPlugin() {
return {
name: 'llms-txt',
async postBuild({siteConfig, routesPaths, routesBuildMetadata, outDir}) {
const fs = await import('node:fs/promises');
const path = await import('node:path');
const lines = [
`# ${siteConfig.title}`,
'',
`> ${siteConfig.tagline}`,
'',
'## Docs',
];
for (const route of routesPaths) {
if (routesBuildMetadata?.[route]?.noIndex) continue;
if (route === '/404.html') continue;
lines.push(`- [${route}](${siteConfig.url}${route})`);
}
await fs.writeFile(
path.join(outDir, 'llms.txt'),
lines.join('\n') + '\n',
);
},
};
},
],
Writing into outDir puts the file at the build root, which is the same place static/ lands, so the baseUrl rule from the table above still applies. Two things worth doing before you ship it: replace the raw route path with a real title, and group routes under separate ## headings rather than one flat list. Agents read the headings.
Route 3: an npm plugin someone already wrote
Two published packages do this, both fetched from the npm registry on 4 September 2026:
| Package | Version | Last published | What it produces |
|---|---|---|---|
docusaurus-plugin-llms | 0.6.0 | 1 September 2026 | An llms.txt index following the llmstxt.org format |
@signalwire/docusaurus-plugin-llms-txt | 1.2.2 | 23 July 2025 | Markdown twins of your HTML pages, plus an llms.txt index |
The second one solves a problem the first does not touch. Your links can point at Markdown rather than HTML, which is what an agent wants at the other end. Read the package README before installing either, and pin the version: both are community packages, not part of @docusaurus/core, which is at 3.10.2 as of this writing.
Choosing which pages to list
Keep it to pages that answer questions. A docs site's route list includes tag pages, archive pages and pagination, and none of those help a model.
- Start with the H1 naming the project. The spec calls this the only required section.
- Follow it with a blockquote summary.
- Group links under
##headings such as## Getting started,## API reference,## Guides. - Give every link a short note after the URL saying what the page covers.
- Leave out anything carrying
noIndex, which is why the hook above readsroutesBuildMetadata.
Confirm which URL you actually shipped
Start with the scan, because it reads the file rather than the status code:
npx aiscan-cli yoursite.com
Read the C2 row. A pass means a file answered at /llms.txt and parsed. If you prefer to check by hand, ask both URLs and compare:
curl -sI https://yoursite.com/llms.txt | head -1
curl -sI https://yoursite.com/your-baseurl/llms.txt | head -1
Expect HTTP/2 200 from whichever one matches your baseUrl. A 404 from the origin root while the subpath returns 200 is the exact failure this guide exists for.
You can trust these status codes on Docusaurus, which is more than you can say for some platforms. Verified on 4 September 2026: docusaurus.io/llms.txt returns a real 404 with a text/html body of 20,038 bytes, and an invented path returns the identical 404. Docusaurus does not answer 200 for files that are absent, so a status check here means something.
Then read the body, not just the header:
curl -s https://yoursite.com/llms.txt | head -20
The first non-blank line must be a single # heading. If you see HTML, you are looking at your 404 page through a redirect.
Where AIScan fits, and where it doesn't
AIScan's C2 check probes /llms.txt at the origin of the URL you give it. Give it the subpath URL and it still asks the origin root, so a spec-legal /docs/llms.txt reads as absent. That is a limitation of the check rather than a fault in your file, and it is on the fix list. What the scan does tell you is whether the file parses, whether the H1 and blockquote are in the right order, and how C2 sits against the rest of the content dimension.
What no scanner can judge is whether the pages you listed are the pages a reader needs. That part is editorial. If you would rather edit something than face a blank file, the llms.txt generator produces a first draft in the right shape, ready to paste into static/llms.txt.
Ship it, then re-scan
Deploy, wait for the build, and run npx aiscan-cli yoursite.com again. Watch the C2 row in the content dimension, and check E1 while you are there, since a host that soft-404s makes every file probe on the report unreliable.
Neighbouring guides worth reading: llms.txt on Hugo covers the other static generator where static/ is the zero-config route, llms.txt on Astro shows the build-time endpoint pattern, and the mistakes that break llms.txt files catalogues what goes wrong inside the file itself. The full rubric for this dimension lives on the content checks page, and every platform guide we publish is indexed at aiscan.site/guides.
Frequently asked questions
My llms.txt is in static/ and the build succeeded, so why does /llms.txt return 404?
Almost certainly baseUrl. Docusaurus copies static/ to the build root, and the build root is served under your baseUrl. If baseUrl is /my-project/, the file answers at /my-project/llms.txt and the origin root has nothing on it. Open docusaurus.config.js, read the baseUrl value, and ask that URL instead. If you need the file at the origin root, either deploy the site at baseUrl: '/' on its own domain or add a rewrite at the host level.
curl returns 200 but the body is HTML instead of my file. What happened?
You are looking at a page, not the file. Two common causes: the request followed a redirect to your docs homepage, or your host rewrites unknown paths to index.html. Run curl without -L to see the raw status, and pipe the body through head -20. The first non-blank line of a valid llms.txt is a single # heading. Anything starting with a doctype means the file is not there.
The postBuild plugin runs without errors but no llms.txt appears in build/. Why?
Check three things in order. First, that the function is inside the plugins array in docusaurus.config.js and returns an object with a name. Second, that you ran a production build; postBuild only fires on docusaurus build, never on docusaurus start. Third, that you wrote to outDir rather than a relative path, because the process working directory is not the build folder.
AIScan reports C2 as a fail but I can open the file in my browser. Is the scan wrong?
If the file lives at a subpath, the scan is looking in the wrong place and this is a known limitation on our side. C2 probes /llms.txt at the origin of the URL you submit. If the file is at the origin root and C2 still fails, the file itself is the problem: check that the content type is text/plain and that the first non-blank line is an H1.
Is a file at /docs/llms.txt valid, or does it have to be at the root?
It is valid. The llms.txt specification allows the file at the root path or at any subpath, and says agents should use the most specific file that applies to a URL. A subpath file is the right choice for a docs site living under a path on a larger domain. Be aware that scanners which probe only the origin root will report it as absent.
Should I use the static folder or a build-time plugin?
Use static/ if your docs set is small and you want editorial control over which pages are listed and how they are described. Use a postBuild hook or an npm plugin once the site is large enough that a hand-written index goes stale between releases. A generated file is always current; a hand-written one is always better written. Some teams generate the link list and keep a hand-written summary at the top.
Does Docusaurus ship an llms.txt output out of the box?
No. There is no built-in llms.txt output and docusaurus.io does not publish one itself, verified on 4 September 2026. The routes are the static folder, your own postBuild hook, or a community npm package. None of them is part of @docusaurus/core, so pin whichever version you install.
Do the links inside llms.txt need to point at Markdown files?
Not required, but it helps. An agent that follows an HTML link has to strip navigation, sidebars and search widgets before it reaches your prose. A Markdown twin skips all of that. Docusaurus does not generate Markdown twins by default, which is the gap the @signalwire package fills: it emits a Markdown version of each page and points the index at those instead of the HTML.
Related guides
The complete AI readiness setup for Squarespace in 2026
Squarespace hands you a finished website and a finished discovery layer at the same time. The robots.txt is written for you, the sitemap is generated for you, and the RSS feed already exists and is…
AI Crawler Traffic in 2026: Only 51.8% of Sites Can Answer "Nothing Changed"
Verified 5 September 2026. All measurements in this article were taken on that date. Every AI crawler that visits your site asks the same handful of questions over and over. Where is your robots.txt.…
The complete AI readiness setup for Framer in 2026
A brandnew Framer site scores better on an agent readiness scan than a new site on almost any other hosted builder, and that is exactly what makes the remaining gaps hard to see. Framer prerenders…
llms.txt vs robots.txt vs sitemap.xml in 2026: Six Files, Six Jobs
Verified 4 September 2026. Every figure below was measured or fetched on that date. Three files keep getting compared as if they were competing for the same job: robots.txt, sitemap.xml and llms.txt.…
