---
title: "CORS on Agent Discovery Files: Why Browser Agents Can't Read Your /.well-known/, llms.txt and OpenAPI"
slug: cors-agent-discovery-files
published: 2026-11-21T13:12:22.281792+00:00
updated: 2026-11-21T13:12:22.281792+00:00
author: "M Asif Rahman"
author_url: https://ar.bd
category: "AI Readiness"
tags: check:P4, check:P2, MCP, CORS, OAuth, well-known, server card
description: "Browser agents get \"Failed to fetch\" on /.well-known/ files that curl reads fine. The CORS headers to send, on 404s too, with nginx, Apache and Next.js config."
url: https://aiscan.site/blog/cors-agent-discovery-files
---

If an agent in a browser tab cannot read your `/.well-known/` files, the file is almost always fine and the response header is missing. Add `Access-Control-Allow-Origin: *` to every public discovery file, on error responses too, and answer the `OPTIONS` preflight with a 2xx. On 21 November 2026, six of the nine discovery URLs we tested on our own site sent no CORS header, and one of them answered 403.

A scanner, Claude or Cursor fetching from a server sends no `Origin` header, so it never sees the problem. A web-based MCP client, an in-page agent or a developer tool running at `http://localhost` does, and all it gets back is `TypeError: Failed to fetch`.

## TL;DR

| File | Should it send `Access-Control-Allow-Origin: *`? | Why |
|---|---|---|
| `/llms.txt`, `/openapi.json`, `/.well-known/api-catalog` | Yes | Public, read-only, no credentials |
| `/.well-known/mcp/server-card.json` (and the card at its catalog URL) | Yes, required | SEP-2127's discovery document says hosted cards MUST |
| `/.well-known/oauth-protected-resource`, `/.well-known/oauth-authorization-server` | Yes | A browser client reads them before it has any token |
| `/.well-known/ai-catalog.json`, `/.well-known/agent-skills/` | Yes | Same reason: public metadata |
| `/mcp` itself, anything behind login | Only for origins you trust | Credentials change the rules |

The fix is one header rule per server, scoped to the discovery paths. AIScan does not test CORS: checks P2 and P4 pass with or without it, so you have to test this by hand.

## What we found on aiscan.site (9 requests, 21 Nov 2026)

We sent each request twice with `curl`: once with `-H 'Origin: https://example.org'`, once without.

| Path | Status with `Origin` | `Access-Control-Allow-Origin` | Status without `Origin` |
|---|---|---|---|
| `/llms.txt` | 200 | `*` | 200 |
| `/openapi.json` | 200 | `*` | 200 |
| `/.well-known/api-catalog` | 200 | `*` | 200 |
| `/.well-known/oauth-authorization-server` | 200 | none | 200 |
| `/.well-known/oauth-protected-resource` | **403** `{"error":"origin not allowed"}` | none | 200 |
| `/.well-known/mcp/server-card.json` | 200 | none | 200 |
| `/.well-known/agent-skills/index.json` | 200 | none | 200 |
| `/.well-known/ai-catalog.json` | 200 | none | 200 |
| `/mcp/server-card` | 404 | none | 404 |

The protected-resource file is the worst case. Our `/mcp` endpoint answers 401 with a `WWW-Authenticate` header that points to it, as [RFC 9728](https://www.rfc-editor.org/rfc/rfc9728) describes. A browser client follows that pointer, sends its own `Origin`, and gets a 403 with no CORS header. RFC 9728 says nothing about CORS, but the file is public metadata a client must read before it can log in. We first saw this on 19 November and filed it with our site team; on 21 November nothing had changed. An `OPTIONS` preflight to our server card returned 200 with no CORS header either. It is our own bug, and this post will be updated when it is fixed.

## What the browser actually does (our lab, Chromium 157)

We served one small JSON file from Python's `http.server` on ports 8002 to 8005 and fetched it from a page on `localhost:8001` with headless Chromium through Playwright. Each row is a separate server setup.

| Server setup | What `fetch()` returned | Console error, verbatim |
|---|---|---|
| No CORS header | `TypeError: Failed to fetch` | "has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource." |
| `*` on every response | 200, JSON parsed | none |
| `*` on every response, file missing | 404, readable | "the server responded with a status of 404" |
| `*` on 2xx only, file missing | `TypeError: Failed to fetch` | the same "No 'Access-Control-Allow-Origin'" line |
| 403 when an `Origin` is present (copy of our bug) | `TypeError: Failed to fetch` | the same line |
| `*` on GET, request sends `If-None-Match` | `TypeError: Failed to fetch` | "Response to preflight request doesn't pass access control check: It does not have HTTP ok status." |
| `*` on GET, request sends `Accept: application/mcp-server-card+json` | 200 | none |

Three results matter. First, a client never learns *why* it failed: the status code is hidden, so a missing file and a missing header look the same. Second, the header has to be on the 404 too, or a client probing several paths cannot tell "not here" from "blocked" and fall back. Third, `If-None-Match` is not a safelisted header, so a client revalidating a cached card sends a preflight `OPTIONS` first, and a server that answers it with 501 or 405 blocks the real request. An `Accept` header does not trigger one.

## Why `*` is right here, and when it is not

According to [MDN's CORS guide](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS), a server that answers a request carrying cookies or HTTP authentication cannot reply with the `*` wildcard: it has to name the origin. That is the line to draw. Discovery files are public and read without credentials, so `*` is safe and simpler. Do not "fix" this by echoing back whatever `Origin` arrives together with `Access-Control-Allow-Credentials: true`. That opens your logged-in endpoints to any site. Keep an origin allowlist on `/mcp` and on account pages if you want one, and keep it off the metadata.

The [SEP-2127 MCP Server Cards](https://modelcontextprotocol.io/seps/2127-mcp-server-cards) extension, which is Final, makes this explicit for cards. Its discovery document, fetched from the extension repository on 21 November 2026, says hosted cards "MUST include appropriate CORS headers to allow browser-based clients" and lists four: `Access-Control-Allow-Origin: *`, `Access-Control-Allow-Methods: GET`, `Access-Control-Allow-Headers: Content-Type, If-None-Match` and `Access-Control-Expose-Headers: ETag`. In its own words, "This is safe because Server Cards contain only public metadata and are read-only." The [Fetch standard's CORS protocol](https://fetch.spec.whatwg.org/#http-cors-protocol) is the normative source for how browsers apply all of it.

## The fix, by server

Scope each rule to the discovery paths. These send the four SEP-2127 headers; trim to the first one for files that are not cards.

**nginx.** In the `location` block for `/.well-known/` (and one each for `/llms.txt` and `/openapi.json`):

```nginx
location /.well-known/ {
    add_header Access-Control-Allow-Origin "*" always;
    add_header Access-Control-Allow-Methods "GET, OPTIONS" always;
    add_header Access-Control-Allow-Headers "Content-Type, If-None-Match" always;
    add_header Access-Control-Expose-Headers "ETag" always;
    if ($request_method = OPTIONS) { return 204; }
}
```

The `always` flag is the part people miss. According to the [nginx headers module documentation](https://nginx.org/en/docs/http/ngx_http_headers_module.html), only with `always` "the header field will be added regardless of the response code"; without it, your 404s go out bare.

**Apache** (`mod_headers`, in the vhost or `.htaccess`):

```apache
<If "%{REQUEST_URI} =~ m#^/(\.well-known/|llms\.txt$|openapi\.json$)#">
    Header always set Access-Control-Allow-Origin "*"
    Header always set Access-Control-Allow-Headers "Content-Type, If-None-Match"
    Header always set Access-Control-Expose-Headers "ETag"
</If>
```

According to the [mod_headers documentation](https://httpd.apache.org/docs/2.4/mod/mod_headers.html), headers in the `always` table "are added to the response even on error".

**Next.js** ([`headers()` in `next.config.js`](https://nextjs.org/docs/app/api-reference/config/next-config-js/headers)):

```js
async headers() {
  return [{
    source: '/.well-known/:path*',
    headers: [
      { key: 'Access-Control-Allow-Origin', value: '*' },
      { key: 'Access-Control-Expose-Headers', value: 'ETag' },
    ],
  }];
}
```

**Netlify and Cloudflare Pages** (a `_headers` file in the publish folder, [Netlify](https://docs.netlify.com/manage/routing/headers/), [Cloudflare](https://developers.cloudflare.com/pages/configuration/headers/)):

```
/.well-known/*
  Access-Control-Allow-Origin: *
/llms.txt
  Access-Control-Allow-Origin: *
```

**Vercel** ([`vercel.json`](https://vercel.com/docs/project-configuration/vercel-json)):

```json
{ "headers": [{ "source": "/.well-known/(.*)",
  "headers": [{ "key": "Access-Control-Allow-Origin", "value": "*" }] }] }
```

For the static hosts, check the 404 response after deploying rather than assuming the rule covers it. If your OAuth metadata lives on an identity provider (ours is on Supabase Auth), the headers come from that provider; set them on the copy or redirect your own origin serves.

## Mistakes that bring the error back

- **Adding the header in the app but serving the file from a CDN edge cache.** The cached copy keeps the old headers until it expires. Purge the discovery paths after the change.
- **Two layers both adding the header.** A proxy and the origin each sending `Access-Control-Allow-Origin` produces a duplicated value, which browsers reject. Set it in one place.
- **A firewall or bot rule that keys on `Origin`.** Our own 403 came from an origin allowlist applied to a file that never needed one. Check WAF rules for the `/.well-known/` paths, not only the server config.
- **Forgetting the redirect.** If `/.well-known/oauth-authorization-server` redirects to your identity provider, the redirect response needs the header as well, and so does the final response.

Who can skip this: a site with no MCP server, no OAuth-protected API and no agent files gains nothing yet. Everyone else should re-test after every server or CDN change, because none of the AIScan checks will catch a regression.

## Verify it

Start with the scan, so you know the files exist before you test headers:

```bash
npx aiscan-cli yoursite.com
```

Check P2 (server card) and P4 (OAuth discovery) in the output. Both should pass, but remember neither one tests CORS. Then send the browser's header by hand, once per file:

```bash
curl -s -o /dev/null -D - -H 'Origin: https://example.org' \
  https://yoursite.com/.well-known/oauth-protected-resource | grep -iE '^(HTTP|access-control)'
# HTTP/2 200
# access-control-allow-origin: *
```

Repeat it on a path that does not exist (you want a 404 that still carries the header), and send the preflight:

```bash
curl -s -o /dev/null -w '%{http_code}\n' -X OPTIONS -H 'Origin: https://example.org' \
  -H 'Access-Control-Request-Method: GET' -H 'Access-Control-Request-Headers: if-none-match' \
  https://yoursite.com/.well-known/mcp/server-card.json
# 204 (any 2xx), with the access-control headers
```

Pass means: a 2xx or a deliberate 404, and `access-control-allow-origin: *`, on every row. Anything else fails in a browser.

For the file locations themselves, see [the server card hub](/blog/publish-mcp-server-card), [OAuth discovery for agents](/blog/oauth-discovery-for-agents), [the protected-resource path fix](/blog/mcp-protected-resource-metadata-path-fix) and [why `/.well-known/` returns 404](/blog/well-known-folder-404-fix). The [capabilities checks](/docs/checks/capabilities) explain P1 to P4, and [every fix guide is on /guides](/guides). Then run a free [AIScan](https://aiscan.site/) of your site and re-test the files above after each deploy.

