---
title: "How to serve text/markdown to AI agents on Docusaurus"
slug: markdown-for-ai-agents-docusaurus
published: 2026-10-04T13:16:34.801652+00:00
updated: 2026-10-04T13:16:34.801652+00:00
author: "AIScan Content Team"
author_url: https://aiscan.site/about
category: "AI Readiness"
tags: check:C1, platform:docusaurus, Docusaurus, Markdown, content negotiation, nginx, AI agents
description: "Make Docusaurus return Markdown to AI agents that send Accept: text/markdown with one plugin and one nginx rule, tested on Docusaurus 3.10.2 and nginx 1.24.0."
url: https://aiscan.site/blog/markdown-for-ai-agents-docusaurus
---

An AI agent that sends `Accept: text/markdown` to a Docusaurus page gets HTML back, because `docusaurus build` writes files and a file cannot read a request header. AIScan check C1 grades exactly that request. This guide makes the build write a Markdown copy of every doc and blog page, then uses one nginx rule to hand that copy to agents at the same URL. I built and ran all of it on Docusaurus 3.10.2, `docusaurus-plugin-llms` 0.6.1 and nginx 1.24.0 on 4 October 2026. Browsers keep getting HTML.

## Quick summary

| If you want to... | Do this | Time |
|---|---|---|
| Write a `.md` file next to every built page | Add `docusaurus-plugin-llms` with `generateMarkdownFiles: true` | 5 min |
| Include blog posts and drop MDX imports | Add `includeBlog` and `excludeImports` | 1 min |
| Serve the copy at the original URL | An nginx `map` on the `Accept` header plus `try_files` | 10 min |
| Confirm it works | `npx aiscan-cli yoursite.com`, then read the C1 row | 1 min |

## Why Docusaurus needs two halves

Check C1 re-requests a page with `Accept: text/markdown` and passes when the server answers `text/markdown`. That is proactive negotiation as defined in [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-proactive-negotiation): one URL, several representations, chosen by a request header. A `/page.md` file alone does not pass, because the scanner asks the original URL.

Docusaurus can supply the first half only with help. It compiles your `.md` and `.mdx` sources into React, so the build output holds `docs/intro/index.html` and no Markdown at all. The [`docusaurus-plugin-llms`](https://www.npmjs.com/package/docusaurus-plugin-llms) package fills that gap, and the web server does the second half. Docusaurus has no request-time code in a static build.

## Step 1: install the plugin

From the project root, run **`npm install docusaurus-plugin-llms`**. It lists `@docusaurus/core` `^3.0.0` as a peer dependency.

## Step 2: write a Markdown copy of every page

Open **`docusaurus.config.js`** and add a `plugins` array next to `presets`:

```js
plugins: [
  ['docusaurus-plugin-llms', {
    generateMarkdownFiles: true,
    includeBlog: true,
    excludeImports: true,
  }],
],
```

Also set `url` to your real site address. The plugin writes absolute links into `llms.txt` from that value, and the scaffold ships with `https://your-docusaurus-site.example.com`.

Run **`npm run build`**. On the scaffold from `create-docusaurus` the log said `Generating individual markdown files...` and `Stats: 13 documents processed`. Each page now has a twin beside its folder:

```text
build/docs/intro/index.html
build/docs/intro.md
build/blog/first-blog-post/index.html
build/blog/first-blog-post.md
```

I read all three option names from the package source. `generateMarkdownFiles` and `includeBlog` both default to `false`, so with the defaults you get `llms.txt` and `llms-full.txt` but no per-page files, and no blog pages at all.

## Step 3: check what an agent will read

Open one generated file. Front matter was gone and the title became a `#` heading. Admonitions stayed as Docusaurus directive syntax (`:::tip[My tip]`), which an agent reads fine. Two things did not convert:

- `excludeImports: true` removed `import` lines, but an inline `export const Highlight = ...` block and the `<Highlight>` tags that use it stayed in the output. Pages that lean on custom MDX components will carry that markup into the Markdown.
- The home page and the `/docs` category pages have no twin. Only pages built from a source file do.

Read your most component-heavy page before you ship. If it is unreadable, the plugin's `preserveComponents` option exists for this case, but I did not test it.

## Step 4: route by Accept header in nginx

Docusaurus builds `docs/intro/index.html` for the route `/docs/intro`, and the twin sits at `/docs/intro.md`. So the rule is "append `.md` to the path when the client prefers Markdown". In the `http` block of **`/etc/nginx/nginx.conf`** or a file it includes, add:

```nginx
types { text/markdown md; }

map $http_accept $md_suffix {
    default "";
    "~*^text/markdown(?!\s*;\s*q=0(\.0+)?(,|$))" ".md";
}

map $uri $bare_uri {
    default $uri;
    "~^(?<p>.+?)/+$" $p;
}
```

Then, in your **`server`** block, point `root` at the `build` folder:

```nginx
add_header Vary Accept always;

location / {
    try_files $bare_uri$md_suffix $uri $uri/index.html =404;
}
error_page 404 /404.html;
```

Reload with **`nginx -t && nginx -s reload`**. The second `map` strips a trailing slash so `/docs/intro/` and `/docs/intro` both find `intro.md`. The first pattern matches only when the header begins with `text/markdown` and that type is not `q=0`. A browser header that starts with `text/html` never matches, and `Vary: Accept` tells caches the same URL has two answers.

## Step 5: send one URL several Accept values

I requested `/docs/intro` with different headers and read the status and content type:

| Accept header | Content-Type returned |
|---|---|
| none | `text/html` |
| `text/markdown` | `text/markdown` |
| `text/markdown, text/html;q=0.8` | `text/markdown` |
| `text/html, text/markdown;q=0.5` | `text/html` |
| `*/*` | `text/html` |
| `text/markdown;q=0` | `text/html` |
| `text/markdown;q=0, text/html` | `text/html` |

All returned 200 with `Vary: Accept`. `/docs/intro/` with a trailing slash and `/docs/tutorial-basics/create-a-page` answered Markdown too, and `/docs/nope` returned 404. The home page `/` answered `text/html` even for `Accept: text/markdown`, which is the gap from Step 3.

## Which route are you on

Pick the branch that matches how the built site is served:

- **You run nginx yourself.** Use Step 4 as written.
- **You sit behind a CDN or edge platform.** Put the same rule in its rewrite layer and keep `Vary: Accept` on both answers. Vercel documents header conditions on rewrites; I did not test it.
- **You publish to GitHub Pages.** The files exist and are linked from `llms.txt`, but C1 cannot pass because the host serves strictly by path.

To find your branch, run the `curl -sI` pair under the check section against your live URL. A `server:` header naming nginx means the first branch applies.

## Two traps I hit

The first is the MIME type. Without `types { text/markdown md; }`, nginx serves `intro.md` as `text/plain` on Ubuntu, and C1 wants `text/markdown`. The second is the host. GitHub Pages and other file-only hosts cannot read the header, so Steps 1 to 3 still produce linked `.md` files but C1 will not pass. Vercel's [`vercel.json` reference](https://vercel.com/docs/project-configuration/vercel-json) documents `has` conditions on a request header, which is the feature a rewrite like this needs. I tested only nginx, so confirm anything else with the commands below.

## Check it: scan first, curl second

Run the scanner against the live site and read the C1 row:

```bash
npx aiscan-cli yoursite.com
```

By hand, request one doc URL twice and compare:

```bash
curl -sI -H "Accept: text/markdown" https://yoursite.com/docs/intro
curl -sI -H "Accept: text/html" https://yoursite.com/docs/intro
```

A pass is `content-type: text/markdown` on the first, `content-type: text/html` on the second, and `vary: accept` on both. If the first still says HTML, the server is not reading the header; start at Step 4. If it returns 404, the `.md` file is missing from `build/`; start at Step 2.

## What this does not cover

The scan makes one request from one place. It cannot see a CDN in front of nginx caching the wrong variant, so purge the cache after the change and confirm `Vary: Accept` survives it. Re-run the scan after upgrading Docusaurus or the plugin, since the plugin is a third-party package at version 0.6.1.

## Next step

Run `npx aiscan-cli yoursite.com` and read C1. The same check on other stacks is in [serving text/markdown on Hugo](https://aiscan.site/blog/markdown-for-ai-agents-hugo) and [on Astro](https://aiscan.site/blog/markdown-for-ai-agents-astro). Docusaurus's other readiness files are in [publishing llms.txt on Docusaurus](https://aiscan.site/blog/llms-txt-docusaurus), and the population-level picture is in [documentation sites and AI agent readiness](https://aiscan.site/blog/documentation-sites-ai-agent-readiness). The check page is [/docs/checks/content](https://aiscan.site/docs/checks/content) and every other fix is in [/guides](https://aiscan.site/guides).