---
title: "Serve Markdown to agents with a Cloudflare rule"
slug: cloudflare-markdown-for-agents-rule
published: 2026-10-28T08:13:25.283151+00:00
updated: 2026-10-28T08:13:25.283151+00:00
author: "M Asif Rahman"
author_url: https://ar.bd
category: "AI Readiness"
tags: check:C1, platform:cloudflare, markdown, content-negotiation, cloudflare, how-to
description: "Turn on Cloudflare Markdown for Agents for a zone or one path, keep browsers on HTML, and test it with three curl requests. Includes a 59-URL measurement."
url: https://aiscan.site/blog/cloudflare-markdown-for-agents-rule
---

To serve Markdown to AI agents from a site on Cloudflare, turn on **Markdown for Agents** for the zone (Dashboard > AI Crawl Control) or for one path with a Configuration Rule. When a client sends `Accept: text/markdown`, Cloudflare fetches your HTML, converts it at the edge and returns `Content-Type: text/markdown` with `Vary: Accept`. A browser, which asks for HTML, still gets HTML. The feature needs a Pro, Business or Enterprise plan. This guide has the dashboard path, the per-path rule, the API call and a pass/fail test, plus what we measured on 28 October 2026 across 59 URLs, our own four sites included.

Written by [M Asif Rahman](https://ar.bd). The adoption data is in our [701-site Markdown study](https://aiscan.site/blog/markdown-twin-beyond-docs); this post is the Cloudflare how-to.

## Quick summary

| If you | Do this | Time | Proof it worked |
|---|---|---|---|
| Want it on for the whole zone | Dashboard > your zone > AI Crawl Control > enable **Markdown for Agents** | 2 min | `curl -H "Accept: text/markdown"` returns `text/markdown` |
| Want it on for `/blog/` or one subdomain only | Configuration Rule with the **Markdown for Agents** setting set to On | 10 min | Rule path returns Markdown, other paths return HTML |
| Manage zones as code | `PATCH` the `content_converter` zone setting | 5 min | Same curl test |
| Are not on Cloudflare, or want exact control | Publish `.md` files and branch on `Accept` at your origin | 30 min | Same curl test, plus `Vary: Accept` |

**The short version:** enable the setting, then test three things: the Markdown request returns Markdown, the browser request still returns HTML, and a missing page does not return a Markdown 200. [Run `npx aiscan-cli yoursite.com`](https://aiscan.site/) first. It grades this under `check:C1` (Markdown content negotiation) and tells you whether you need to do anything. AIScan cannot tell you whether the Markdown came from Cloudflare's edge or from your origin, and it does not grade the `Vary` header, so the curl test below covers both.

## What happens at the edge

Content negotiation is old HTTP. The client lists the formats it can use in the `Accept` header, the server picks one, and `Vary: Accept` tells caches that the same URL has more than one version. [MDN's content negotiation guide](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation) and [RFC 9110 section 12](https://www.rfc-editor.org/rfc/rfc9110.html#name-proactive-negotiation) describe it. Markdown for Agents applies it to HTML pages: Cloudflare asks your origin for the normal HTML, strips non-content elements (navigation, headers, footers, scripts, styles), converts the rest to Markdown and streams it back.

Think of a restaurant that keeps one kitchen and two menus. The kitchen (your origin) cooks the same dishes. The waiter (Cloudflare) hands a picture menu to people who walk in and a plain text menu to the delivery bots that ask for one. You do not maintain a second menu.

Cloudflare's [Markdown for Agents documentation](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) (last updated 9 October 2026 when we read it) describes the output. The converted response begins with YAML frontmatter built from your `<meta>` tags (`title`, `description` and `image`, with Open Graph tags as fallbacks), then the body, then any JSON-LD as a fenced `json` block at the end. Three consequences matter for a site owner:

- **The converted response drops `ETag` and `Last-Modified`.** Cloudflare says conditional requests cannot be honoured on converted responses. If you set up validators from our [ETag and Last-Modified guide](https://aiscan.site/blog/conditional-requests-etag-last-modified), they still work for the HTML; they do not carry over to the Markdown.
- **A default Content Signal is added.** If your origin sends no `content-signal` header, the converted response carries `Content-Signal: ai-train=yes, search=yes, ai-input=yes`. If you want a different policy, set the header at your origin; Cloudflare keeps yours.
- **There is a size limit.** Conversion stops at 6 MiB of decompressed HTML, and only HTML is converted.

## What we measured

On 28 October 2026 we sent three requests to each of 59 URLs: `Accept: text/markdown`, a browser-style `Accept` header, and `Accept: text/markdown;q=0, text/html`. The URLs were our four sites, Cloudflare's own sites and a spread of developer-documentation and SaaS sites. 53 answered 200; the other six (a 400, three 403s, a 404 and a 429) are left out.

| Measure (n = 53 URLs answering 200) | Count |
|---|---|
| Returned `text/markdown` to `Accept: text/markdown` | 21 (40%) |
| Of those, sent `Vary: Accept` | 19 of 21 |
| Behind Cloudflare (`server: cloudflare`) | 20 |
| Behind Cloudflare and returned Markdown | 7 of 20 |
| Not behind Cloudflare and returned Markdown | 14 of 33 |
| Returned HTML to a browser `Accept` header | 53 of 53 |
| Returned Markdown even when `text/markdown;q=0` was sent | 9 of 21 |

Two things stand out. First, serving Markdown is not a Cloudflare-only habit: 14 sites that are not behind Cloudflare do it, so we cannot say how many of the seven Cloudflare-fronted hits use the Markdown for Agents setting rather than their own code. Second, the `Accept` header is easy to handle badly. Nine of the 21 URLs ignored an explicit `q=0` (RFC 9110 says that means "not acceptable"), and two sent Markdown without `Vary: Accept`, which lets a shared cache hand the Markdown copy to a browser.

| Our sites (28 Oct 2026) | `Accept: text/markdown` result |
|---|---|
| aiscan.site | `text/markdown` with `Vary: Accept` (ignores `q=0`, noted for a fix) |
| storeseo.com (Cloudflare) | HTML |
| bettershield.ai (Cloudflare) | HTML |
| xspeedcache.com | HTML |

Two of our Cloudflare-fronted sites returned HTML. We read that as "not turned on", not as a fault, because the scan can still pass without negotiation: AIScan's `check:C1` on storeseo.com passes through `/index.md` returning `text/markdown`. A fixed-path file counts. Negotiation is the more convenient route for agents that send the header; the file is the route that works without any CDN feature.

On Cloudflare's own docs site the picture is what the documentation promises: the `Accept: text/markdown` request got `text/markdown; charset=utf-8` with `Vary: accept-encoding, accept`, no `ETag` and no `Last-Modified` on the Markdown response. `Accept: */*` and the browser header got HTML. `text/markdown, text/html;q=0.5` got Markdown, `text/html, text/markdown;q=0.9` got HTML, and `q=0` on Markdown got HTML, so Cloudflare weighs the q-values. A made-up path returned a 404 in Markdown, not a 200.

## Path 1: turn it on for the whole zone

1. Log in to the Cloudflare dashboard and select your account. You need a Pro or Business plan; the docs also list Enterprise and SSL for SaaS.
2. Select the zone (the domain) you want to change.
3. Open **AI Crawl Control**.
4. Enable **Markdown for Agents**.
5. Run the test at the end of this guide against a normal page.

Menu names in Cloudflare's dashboard move between releases, and its own docs use both "Security" and "Crawlers" for neighbouring tabs. If you cannot find the toggle, use the API call in Path 3, which does not depend on the layout.

## Path 2: turn it on for one subdomain or path

Use this when only the blog or docs should be served as Markdown, and checkout, account and app pages should not be.

1. Select the zone, then go to **Rules > Overview** and choose **Create rule > Configuration Rules**.
2. Under **When incoming requests match**, build an expression. For a subdomain: `http.host eq "docs.example.com"`. For a path: `starts_with(http.request.uri.path, "/blog/")`.
3. Under **Then the settings are**, choose **Add setting > Markdown for Agents** and set it to **On**.
4. Select **Deploy**.
5. Test one URL inside the rule and one outside it. The outside URL should return HTML to both requests.

## Path 3: the API, for zones managed as code

Send a `PATCH` to the `content_converter` zone setting with a token that has Zone Settings edit permission:

```bash
curl -X PATCH "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/settings/content_converter" \
  --header "Content-Type: application/json" \
  --header "Authorization: Bearer $API_TOKEN" \
  --data '{"value": "on"}'
```

For a per-path rule, `PUT` a ruleset to the `http_config_settings` phase with `"action": "set_config"` and `"content_converter": true` in `action_parameters`. Cloudflare's docs show the full body, and we did not run the dashboard or API steps against a zone of our own, so check them against the current page before you rely on them.

## If you are not on Cloudflare, or want to own the output

The edge conversion works from the HTML your origin returns, and only HTML. If you can generate Markdown yourself (from the source you already render), publish a `.md` twin at a fixed path and branch on `Accept` at your origin. Return `Content-Type: text/markdown; charset=utf-8` and `Vary: Accept`, and give any unknown path a real 404. Our [Markdown twin study](https://aiscan.site/blog/markdown-twin-beyond-docs) has the nginx and Apache snippets, and the [curl walkthrough](https://aiscan.site/blog/fetch-site-as-ai-agent-curl) shows how an agent sees the result.

## Verify: AIScan first, curl second

Run `npx aiscan-cli yoursite.com` or paste the URL at [aiscan.site](https://aiscan.site/). Look at `check:C1`. A pass means a Markdown route was found, either through negotiation or a file such as `/index.md`. Then confirm the details AIScan does not cover:

```bash
U=https://example.com/blog/some-post/
# 1. Markdown request
curl -s -o /dev/null -D - -H "Accept: text/markdown" "$U" | grep -i -E '^(HTTP|content-type|vary|content-signal)'
# 2. Browser request must still be HTML
curl -s -o /dev/null -D - -H "Accept: text/html,application/xhtml+xml,*/*;q=0.8" "$U" | grep -i -E '^(HTTP|content-type)'
# 3. A missing page must not be a Markdown 200
curl -s -o /dev/null -w '%{http_code} %{content_type}\n' -H "Accept: text/markdown" "https://example.com/no-such-page-zz/"
```

| Result | Meaning | First fix |
|---|---|---|
| Request 1 returns `text/html` | Not enabled for this host or path | Re-check the toggle or the rule expression |
| Request 1 is Markdown but `Vary` lacks `Accept` | A cache can serve the wrong copy | Your own origin code, not the edge setting; add the header |
| Request 2 returns Markdown | Browsers would get Markdown | Check for a rule that forces conversion for every request |
| Request 3 returns `200 text/markdown` | Soft 404 | Fix the origin's 404 handling |
| Frontmatter has no `description` | No `<meta name="description">` in the HTML | Add the tag; the converter reads it |

## Who can skip this

If your site already passes `check:C1` through a Markdown file you maintain, turning on the edge setting adds a second source of truth and gives you two outputs to keep consistent. Sites that sell access to their content should read the policy section above first, since the default `ai-train=yes` signal appears only when your origin sets nothing. And if the free plan is what you run, publish the `.md` twin instead.

## Next steps

Run `npx aiscan-cli yoursite.com` to see where `check:C1` stands, enable the setting for the paths you want agents to read, and run the three-request test. The [content checks page](https://aiscan.site/docs/checks/content) explains C1 and its neighbours, the [AI Crawl Control setup guide](https://aiscan.site/blog/cloudflare-ai-crawl-control-setup) covers the crawler side of the same dashboard, and the [guides](https://aiscan.site/guides) list the rest.