---
title: "How to serve text/markdown to AI agents on Next.js"
slug: markdown-for-ai-agents-nextjs
published: 2026-09-30T13:16:41.148103+00:00
updated: 2026-10-04T12:03:18.317448+00:00
author: "AIScan Content Team"
author_url: https://aiscan.site/about
category: "AI Readiness"
tags: check:C1, platform:nextjs, Next.js, Markdown, content negotiation, Route Handlers, AI agents
description: "Return Markdown to agents that send Accept: text/markdown on Next.js with one beforeFiles rewrite and one Route Handler, tested live on Next.js 16.3.7."
url: https://aiscan.site/blog/markdown-for-ai-agents-nextjs
---

An agent that sends `Accept: text/markdown` to a Next.js site gets the full HTML shell back unless you build the second representation yourself. Next.js does not do content negotiation for you. This guide adds it with one rewrite rule and one Route Handler, which is about 25 lines of code, and the test app behind it returned Markdown to the agent and HTML to the browser on the same URL. AIScan check C1 grades exactly that behaviour.

## Quick summary

| If you want to... | Do this | Time |
|---|---|---|
| Serve Markdown on the same URL | A `beforeFiles` rewrite in `next.config.ts` that matches the `Accept` header | 5 min |
| Produce the Markdown | A Route Handler at `app/md/blog/[slug]/route.ts` | 10 min |
| Keep caches honest | `Vary: Accept` on the Markdown response, and on the HTML at your CDN | 5 min |
| Confirm it works | `npx aiscan-cli yoursite.com`, look for C1 | 1 min |

Next.js's own docs are the model. Fetched on 30 September 2026, `nextjs.org/docs/app/getting-started/installation` with `Accept: text/markdown` answers 200, `content-type: text/markdown; charset=utf-8`, `vary: Accept`, and an `x-matched-path` of `/docs/llm-digest/app/getting-started/installation`. That header shows a rewrite to a separate route, which is the pattern below.

## Why a rewrite and not a `.md` URL

A page component cannot change the content type, because a page always renders HTML. A Route Handler can return any `Response`, so it is the right home for the Markdown. A rewrite then points the agent's request at that handler without changing the URL the agent asked for. The [Next.js rewrites reference](https://nextjs.org/docs/app/api-reference/config/next-config-js/rewrites) documents the `has` field: the rewrite applies only when a header, cookie, host or query value also matches.

`beforeFiles` matters. Per the same page, `beforeFiles` rewrites are checked before the filesystem and dynamic routes, so the rewrite wins over `app/blog/[slug]/page.tsx`. An `afterFiles` rewrite would never fire, because the page already matches.

## Step 1: add the rewrite

1. Open **next.config.ts** in the project root.
2. Add a `rewrites()` function that returns an object with a `beforeFiles` array:

```ts
import type { NextConfig } from 'next'

const config: NextConfig = {
  async rewrites() {
    return {
      beforeFiles: [
        {
          source: '/blog/:slug',
          has: [{ type: 'header', key: 'accept', value: '(.*)text/markdown(.*)' }],
          destination: '/md/blog/:slug',
        },
      ],
    }
  },
}
export default config
```

The `value` is a regex-like string, so `(.*)text/markdown(.*)` matches `text/markdown` anywhere in the header, including `text/markdown, text/html;q=0.5`. Repeat the rule for each section of the site you want covered, for example `/docs/:path*`.

## Step 2: add the Route Handler

1. Create the folder **app/md/blog/[slug]/** and add **route.ts** inside it.
2. Return Markdown with the right type and a `Vary` header:

```ts
export async function GET(
  _req: Request,
  { params }: { params: Promise<{ slug: string }> },
) {
  const { slug } = await params
  const body = await getPostAsMarkdown(slug) // your own loader
  if (!body) return new Response('Not found', { status: 404 })
  return new Response(body, {
    headers: {
      'Content-Type': 'text/markdown; charset=utf-8',
      Vary: 'Accept',
    },
  })
}
```

`getPostAsMarkdown` is yours. If your posts already live as MDX or Markdown files, read the file and strip the frontmatter. If they come from a CMS, convert the rich text to Markdown here. Return a real 404 for a slug that does not exist, not a 200 with an error page, or a scanner cannot tell a working route from a catch-all.

## Step 3: handle caching

Two caches can serve the wrong representation: a CDN in front of the site and any shared proxy. The fix is `Vary: Accept`, which tells a cache that the same URL has more than one version.

In our test on Next.js 16.3.7 with `next start`, the Markdown response carried `vary: Accept` from the Route Handler. The HTML response did not. Its `Vary` listed the RSC headers and `Accept-Encoding`, and adding a `headers()` entry in `next.config.ts` or appending the header in `proxy.ts` did not change that. So set the cache key at the layer that caches: add `Accept` to the cache key or `Vary` handling in your CDN or host settings. If you skip that, a CDN that cached the HTML first will hand HTML to the next agent.

## Which branch are you on?

| Your setup | What to do |
|---|---|
| Self-hosted `next start` behind Nginx or Cloudflare | Rewrite plus CDN `Vary: Accept` setting |
| Vercel or another managed host | Same rewrite and handler. Check the response headers before you assume the platform varies on `Accept` |
| Static export (`output: 'export'`) | Rewrites need a server, so this pattern does not apply. Generate `.md` files at build time instead |

To find your branch, run the verification below and read the `vary` line.

## When it does not work

| Symptom | Cause | Fix |
|---|---|---|
| Agent still gets HTML | The rewrite is in `afterFiles`, or `source` does not match the real path | Move it to `beforeFiles` and compare `source` with the URL you request |
| Browser shows Markdown | A cache stored the Markdown under the plain URL | Add `Accept` to the cache key, then purge that path |
| Every URL returns Markdown | `source` is a broad pattern and the handler never returns 404 | Narrow `source` and return 404 for unknown slugs |

After changing **next.config.ts**, rebuild and restart the server before you test.

## Check it: scan first, curl second

Run the scan. It is free and needs no account:

```
npx aiscan-cli yoursite.com
```

The result lists **C1, Markdown content negotiation** under [content](https://aiscan.site/docs/checks/content) with the evidence string it saw. You can also paste the URL into [the scanner](https://aiscan.site/).

If you would rather check by hand:

```bash
U=https://yoursite.com/blog/some-post
curl -sI -H 'Accept: text/markdown' $U | grep -i -E '^(HTTP|content-type|vary)'
curl -sI -H 'Accept: text/html'     $U | grep -i -E '^(HTTP|content-type|vary)'
curl -sI -H 'Accept: text/markdown' $U/no-such-post-9137 | head -1
```

Expected: the first returns `200` and `content-type: text/markdown; charset=utf-8`. The second returns `text/html`. The third returns `404`. If the third returns 200, your handler is a catch-all and C1 cannot be trusted. Our [documentation sites study](https://aiscan.site/blog/documentation-sites-ai-agent-readiness) found 21 of 52 sites answering `Accept: text/markdown` with HTTP 200 and a non-Markdown type, which is the failure this control request catches.

## Limits and maintenance

Serving Markdown does nothing if the HTML is an empty shell, so read [React apps that go invisible to crawlers](https://aiscan.site/blog/nextjs-react-invisible-to-ai-crawlers) too. Point your [llms.txt](https://aiscan.site/blog/llms-txt-nextjs) links at pages that now have a Markdown form. AIScan fetches with its own identity, so it cannot show what a specific crawler user agent receives. Recheck after any Next.js upgrade that touches routing, and after every new section you add.

More platform walkthroughs are in the [guides index](https://aiscan.site/guides). The Next.js hub is at [/docs/platforms/nextjs](https://aiscan.site/docs/platforms/nextjs). Run `npx aiscan-cli yoursite.com` now and read C1.