How to serve text/markdown to AI agents on Next.js

An agent that sends Accept: text/markdown to a Next.js site gets the full HTML shell back unless you build the second representation yourself. Next.js does not do content negotiation for you. This guide adds it with one rewrite rule and one Route Handler, which is about 25 lines of code, and the test app behind it returned Markdown to the agent and HTML to the browser on the same URL. AIScan check C1 grades exactly that behaviour.

Quick summary

Next.js's own docs are the model. Fetched on 30 September 2026, nextjs.org/docs/app/getting-started/installation with Accept: text/markdown answers 200, content-type: text/markdown; charset=utf-8, vary: Accept, and an x-matched-path of /docs/llm-digest/app/getting-started/installation. That header shows a rewrite to a separate route, which is the pattern below.

Why a rewrite and not a .md URL

A page component cannot change the content type, because a page always renders HTML. A Route Handler can return any Response, so it is the right home for the Markdown. A rewrite then points the agent's request at that handler without changing the URL the agent asked for. The Next.js rewrites reference documents the has field: the rewrite applies only when a header, cookie, host or query value also matches.

beforeFiles matters. Per the same page, beforeFiles rewrites are checked before the filesystem and dynamic routes, so the rewrite wins over app/blog/[slug]/page.tsx. An afterFiles rewrite would never fire, because the page already matches.

Step 1: add the rewrite

1. Open next.config.ts in the project root.
2. Add a rewrites() function that returns an object with a beforeFiles array:

import type { NextConfig } from 'next'

const config: NextConfig = {
  async rewrites() {
    return {
      beforeFiles: [
        {
          source: '/blog/:slug',
          has: [{ type: 'header', key: 'accept', value: '(.*)text/markdown(.*)' }],
          destination: '/md/blog/:slug',
        },
      ],
    }
  },
}
export default config

The value is a regex-like string, so (.*)text/markdown(.*) matches text/markdown anywhere in the header, including text/markdown, text/html;q=0.5. Repeat the rule for each section of the site you want covered, for example /docs/:path*.

Step 2: add the Route Handler

1. Create the folder app/md/blog/[slug]/ and add route.ts inside it.
2. Return Markdown with the right type and a Vary header:

export async function GET(
  _req: Request,
  { params }: { params: Promise<{ slug: string }> },
) {
  const { slug } = await params
  const body = await getPostAsMarkdown(slug) // your own loader
  if (!body) return new Response('Not found', { status: 404 })
  return new Response(body, {
    headers: {
      'Content-Type': 'text/markdown; charset=utf-8',
      Vary: 'Accept',
    },
  })
}

getPostAsMarkdown is yours. If your posts already live as MDX or Markdown files, read the file and strip the frontmatter. If they come from a CMS, convert the rich text to Markdown here. Return a real 404 for a slug that does not exist, not a 200 with an error page, or a scanner cannot tell a working route from a catch-all.

Step 3: handle caching

Two caches can serve the wrong representation: a CDN in front of the site and any shared proxy. The fix is Vary: Accept, which tells a cache that the same URL has more than one version.

In our test on Next.js 16.3.7 with next start, the Markdown response carried vary: Accept from the Route Handler. The HTML response did not. Its Vary listed the RSC headers and Accept-Encoding, and adding a headers() entry in next.config.ts or appending the header in proxy.ts did not change that. So set the cache key at the layer that caches: add Accept to the cache key or Vary handling in your CDN or host settings. If you skip that, a CDN that cached the HTML first will hand HTML to the next agent.

Which branch are you on?

To find your branch, run the verification below and read the vary line.

When it does not work

After changing next.config.ts, rebuild and restart the server before you test.

Check it: scan first, curl second

Run the scan. It is free and needs no account:

npx aiscan-cli yoursite.com

The result lists C1, Markdown content negotiation under content with the evidence string it saw. You can also paste the URL into the scanner.

If you would rather check by hand:

U=https://yoursite.com/blog/some-post
curl -sI -H 'Accept: text/markdown' $U | grep -i -E '^(HTTP|content-type|vary)'
curl -sI -H 'Accept: text/html'     $U | grep -i -E '^(HTTP|content-type|vary)'
curl -sI -H 'Accept: text/markdown' $U/no-such-post-9137 | head -1

Expected: the first returns 200 and content-type: text/markdown; charset=utf-8. The second returns text/html. The third returns 404. If the third returns 200, your handler is a catch-all and C1 cannot be trusted. Our documentation sites study found 21 of 52 sites answering Accept: text/markdown with HTTP 200 and a non-Markdown type, which is the failure this control request catches.

Limits and maintenance

Serving Markdown does nothing if the HTML is an empty shell, so read React apps that go invisible to crawlers too. Point your llms.txt links at pages that now have a Markdown form. AIScan fetches with its own identity, so it cannot show what a specific crawler user agent receives. Recheck after any Next.js upgrade that touches routing, and after every new section you add.

More platform walkthroughs are in the guides index. The Next.js hub is at /docs/platforms/nextjs. Run npx aiscan-cli yoursite.com now and read C1.