How to serve text/markdown to AI agents on Docusaurus

An AI agent that sends Accept: text/markdown to a Docusaurus page gets HTML back, because docusaurus build writes files and a file cannot read a request header. AIScan check C1 grades exactly that request. This guide makes the build write a Markdown copy of every doc and blog page, then uses one nginx rule to hand that copy to agents at the same URL. I built and ran all of it on Docusaurus 3.10.2, docusaurus-plugin-llms 0.6.1 and nginx 1.24.0 on 4 October 2026. Browsers keep getting HTML.

Quick summary

Why Docusaurus needs two halves

Check C1 re-requests a page with Accept: text/markdown and passes when the server answers text/markdown. That is proactive negotiation as defined in RFC 9110: one URL, several representations, chosen by a request header. A /page.md file alone does not pass, because the scanner asks the original URL.

Docusaurus can supply the first half only with help. It compiles your .md and .mdx sources into React, so the build output holds docs/intro/index.html and no Markdown at all. The docusaurus-plugin-llms package fills that gap, and the web server does the second half. Docusaurus has no request-time code in a static build.

Step 1: install the plugin

From the project root, run npm install docusaurus-plugin-llms. It lists @docusaurus/core ^3.0.0 as a peer dependency.

Step 2: write a Markdown copy of every page

Open docusaurus.config.js and add a plugins array next to presets:

plugins: [
  ['docusaurus-plugin-llms', {
    generateMarkdownFiles: true,
    includeBlog: true,
    excludeImports: true,
  }],
],

Also set url to your real site address. The plugin writes absolute links into llms.txt from that value, and the scaffold ships with https://your-docusaurus-site.example.com.

Run npm run build. On the scaffold from create-docusaurus the log said Generating individual markdown files... and Stats: 13 documents processed. Each page now has a twin beside its folder:

build/docs/intro/index.html
build/docs/intro.md
build/blog/first-blog-post/index.html
build/blog/first-blog-post.md

I read all three option names from the package source. generateMarkdownFiles and includeBlog both default to false, so with the defaults you get llms.txt and llms-full.txt but no per-page files, and no blog pages at all.

Step 3: check what an agent will read

Open one generated file. Front matter was gone and the title became a # heading. Admonitions stayed as Docusaurus directive syntax (:::tip[My tip]), which an agent reads fine. Two things did not convert:

• excludeImports: true removed import lines, but an inline export const Highlight = ... block and the <Highlight> tags that use it stayed in the output. Pages that lean on custom MDX components will carry that markup into the Markdown.
• The home page and the /docs category pages have no twin. Only pages built from a source file do.

Read your most component-heavy page before you ship. If it is unreadable, the plugin's preserveComponents option exists for this case, but I did not test it.

Step 4: route by Accept header in nginx

Docusaurus builds docs/intro/index.html for the route /docs/intro, and the twin sits at /docs/intro.md. So the rule is "append .md to the path when the client prefers Markdown". In the http block of /etc/nginx/nginx.conf or a file it includes, add:

types { text/markdown md; }

map $http_accept $md_suffix {
    default "";
    "~*^text/markdown(?!\s*;\s*q=0(\.0+)?(,|$))" ".md";
}

map $uri $bare_uri {
    default $uri;
    "~^(?<p>.+?)/+$" $p;
}

Then, in your server block, point root at the build folder:

add_header Vary Accept always;

location / {
    try_files $bare_uri$md_suffix $uri $uri/index.html =404;
}
error_page 404 /404.html;

Reload with nginx -t && nginx -s reload. The second map strips a trailing slash so /docs/intro/ and /docs/intro both find intro.md. The first pattern matches only when the header begins with text/markdown and that type is not q=0. A browser header that starts with text/html never matches, and Vary: Accept tells caches the same URL has two answers.

Step 5: send one URL several Accept values

I requested /docs/intro with different headers and read the status and content type:

All returned 200 with Vary: Accept. /docs/intro/ with a trailing slash and /docs/tutorial-basics/create-a-page answered Markdown too, and /docs/nope returned 404. The home page / answered text/html even for Accept: text/markdown, which is the gap from Step 3.

Which route are you on

Pick the branch that matches how the built site is served:

• You run nginx yourself. Use Step 4 as written.
• You sit behind a CDN or edge platform. Put the same rule in its rewrite layer and keep Vary: Accept on both answers. Vercel documents header conditions on rewrites; I did not test it.
• You publish to GitHub Pages. The files exist and are linked from llms.txt, but C1 cannot pass because the host serves strictly by path.

To find your branch, run the curl -sI pair under the check section against your live URL. A server: header naming nginx means the first branch applies.

Two traps I hit

The first is the MIME type. Without types { text/markdown md; }, nginx serves intro.md as text/plain on Ubuntu, and C1 wants text/markdown. The second is the host. GitHub Pages and other file-only hosts cannot read the header, so Steps 1 to 3 still produce linked .md files but C1 will not pass. Vercel's vercel.json reference documents has conditions on a request header, which is the feature a rewrite like this needs. I tested only nginx, so confirm anything else with the commands below.

Check it: scan first, curl second

Run the scanner against the live site and read the C1 row:

npx aiscan-cli yoursite.com

By hand, request one doc URL twice and compare:

curl -sI -H "Accept: text/markdown" https://yoursite.com/docs/intro
curl -sI -H "Accept: text/html" https://yoursite.com/docs/intro

A pass is content-type: text/markdown on the first, content-type: text/html on the second, and vary: accept on both. If the first still says HTML, the server is not reading the header; start at Step 4. If it returns 404, the .md file is missing from build/; start at Step 2.

What this does not cover

The scan makes one request from one place. It cannot see a CDN in front of nginx caching the wrong variant, so purge the cache after the change and confirm Vary: Accept survives it. Re-run the scan after upgrading Docusaurus or the plugin, since the plugin is a third-party package at version 0.6.1.

Next step

Run npx aiscan-cli yoursite.com and read C1. The same check on other stacks is in serving text/markdown on Hugo and on Astro. Docusaurus's other readiness files are in publishing llms.txt on Docusaurus, and the population-level picture is in documentation sites and AI agent readiness. The check page is /docs/checks/content and every other fix is in /guides.