---
title: "Why /.well-known/ files return 404 or 403 on your host, and how to fix it"
slug: well-known-folder-404-fix
published: 2026-10-26T13:22:33.432946+00:00
updated: 2026-10-26T13:22:33.432946+00:00
author: "M Asif Rahman"
author_url: https://ar.bd
category: "AI Readiness"
tags: well-known, dotfiles, GitHub Pages, nginx, Apache, Firebase Hosting, agent discovery, AI readiness, check:P1, check:P2, check:P3, check:P4, platform:nginx, platform:github-pages
description: "Fix /.well-known/ returning 404 or 403: Jekyll drops it, nginx and Apache dotfile rules block it. Tested fixes and a curl check for each host."
url: https://aiscan.site/blog/well-known-folder-404-fix
---

Your `/.well-known/` file is returning 404 or 403 because something between your repository and the browser treats a folder that starts with a dot as private. In our tests on 26 October 2026, a default Jekyll build dropped the folder, and a stock nginx rule that blocks dotfiles returned 403 for it. Firebase Hosting, which is often blamed, uploaded it in our file-listing test. Each of the four fixes below is one or two lines.

By [M Asif Rahman](https://ar.bd). Verified on 26 October 2026 with Jekyll 4.4.1, firebase-tools 15.34.1, nginx 1.24.0 and Apache 2.4.58.

## TL;DR: which host drops the folder, and the fix

| Host or server | Default result (tested 26 Oct 2026) | Fix | Status after fix |
|---|---|---|---|
| GitHub Pages (Jekyll build) | File missing from `_site`, so 404 | Add `include: [".well-known"]` to `_config.yml`, or an empty `.nojekyll` file | File built (tested for `include`) |
| nginx with `location ~ /\. { deny all; }` | 403 | `location ^~ /.well-known/ { allow all; }` or `location ~ /\.(?!well-known) { deny all; }` | 200 |
| Apache with `RedirectMatch 404 /\..*$` | 404 | `RedirectMatch 404 "/\.(?!well-known)"` | 200 |
| Firebase Hosting, default `firebase.json` | Uploaded (file-listing test, no live deploy) | None needed | Uploaded |

**Start here:** run `npx aiscan-cli yoursite.com` and read the lines for P1, P2, P3 and P4, the [capabilities checks](https://aiscan.site/docs/checks/capabilities) that read a `/.well-known/` path. If you would rather check by hand, `curl -I https://yoursite.com/.well-known/mcp/server-card.json` and look for `200`.

## Why a dot-folder disappears

[RFC 8615](https://www.rfc-editor.org/rfc/rfc8615.txt) (Nottingham, May 2019, fetched from the RFC Editor on 26 October 2026) defines `/.well-known/` as the path prefix for well-known locations. Agent discovery files live there: the API catalog, the OAuth metadata and the MCP server card from our [server card guide](https://aiscan.site/blog/publish-mcp-server-card). The leading dot is the problem. Unix tools, static site generators and server rules all use a leading dot to mean "hidden, do not serve". A rule written to hide `.git` and `.env` catches `.well-known` in the same sweep.

So the file is usually not missing from your repository. It is dropped at build time, skipped at upload, or refused at request time. Find which stage by checking the same file in three places: your repository, your build output, and the live URL.

AIScan reads fixed paths. It can tell you the file is absent, but not whether your host stripped it or you never created it. The stage check above is how you tell.

## GitHub Pages: the Jekyll build skips it

According to the [Jekyll configuration docs](https://jekyllrb.com/docs/configuration/options/), files and folders that start with a dot are excluded unless you list them under `include`. We built a test site with `.well-known/mcp/server-card.json` and ran `jekyll build`:

| Config | `_site/.well-known/mcp/server-card.json` |
|---|---|
| Default `_config.yml` | Not built |
| `include: [".well-known"]` | Built |

Fix it in **_config.yml** by adding `include: [".well-known"]`. The other route is an empty file named `.nojekyll` in the root of the published branch. According to [GitHub's own post on bypassing Jekyll](https://github.blog/news-insights/the-library/bypassing-jekyll-on-github-pages/), that file turns Jekyll processing off for the site. We could not test `.nojekyll` locally, because a local `jekyll build` ignores it; it only matters to GitHub's pipeline. Prefer `include` if you want a result you can reproduce on your own machine.

If you deploy with a GitHub Actions workflow that uploads a build folder, check the folder itself: some upload steps skip hidden files. Open the artifact and look for `.well-known` before blaming the host.

## nginx: the dotfile deny rule

Many nginx configs carry a rule that hides dotfiles. It protects `.env` and `.git`, and it also catches `/.well-known/`. We ran four configs against the same folder:

| nginx config | `/.well-known/mcp/server-card.json` | `/.env` | `/.git/HEAD` |
|---|---|---|---|
| `location ~ /\. { deny all; }` | 403 | 403 | 403 |
| `location ^~ /.well-known/ { allow all; }` plus the deny rule | 200 | 403 | 403 |
| `location ~ /\.(?!well-known) { deny all; }` | 200 | 403 | 403 |
| No dotfile rule | 200 | 200 | 200 |

The last row is the reason not to delete the rule. Without it, `.env` and `.git/HEAD` were served too. Keep the deny and carve out the one prefix. The `^~` modifier works because a matching prefix location with it stops nginx from checking regular-expression locations. According to the [nginx location documentation](https://nginx.org/en/docs/http/ngx_http_core_module.html#location), a longest matching prefix location with the modifier stops the regular-expression search. Reload with `nginx -t && nginx -s reload`, then re-run your `curl`.

If you manage servers on [xCloud](https://xcloud.host), the same location block goes into the site's custom Nginx configuration.

## Apache: the dotfile redirect

A common hardening line is `RedirectMatch 404 /\..*$`. It returned 404 for the server card and for `.env`. Replacing it with `RedirectMatch 404 "/\.(?!well-known)"` returned 200 for the card and kept `.env` at 404. With no rule at all, `.env` returned 200.

## Firebase Hosting: not the culprit in our test

The [Firebase Hosting docs](https://firebase.google.com/docs/hosting/full-config), fetched on 26 October 2026, list `**/.*` in the default `ignore` array and describe it as matching any file beginning with a dot. We expected it to block the folder. We ran the `listFiles` function that `firebase deploy` uses, from firebase-tools 15.34.1, against a `public/` folder with the default `ignore` array:

| File in `public/` | Uploaded with default `ignore` |
|---|---|
| `.env` | No |
| `.well-known/direct.json` | Yes |
| `.well-known/mcp/server-card.json` | Yes |

The pattern skips files whose own name starts with a dot, not files inside a dot-folder. It would block the folder if you added `**/.*/**` to `ignore`. If a Firebase deploy loses your card, search `firebase.json` for that pattern first. We did not run a live deploy, only the file listing.

## How common is the problem?

Our [agent discovery census](https://aiscan.site/blog/well-known-agent-files-corpus) of 835 scanned hosts found that 301 pass at least one of the four well-known checks and 534 pass none. Those counts do not say why a host fails. A missing file and a stripped file look identical, so some share of the 534 is probably this bug, and we cannot size it from stored scan results.

## Verify the fix

Run the scan first: `npx aiscan-cli yoursite.com`, then read P1 to P4. By hand, the equivalent is:

```bash
curl -sI https://yoursite.com/.well-known/mcp/server-card.json | head -n 1
```

Expected: `HTTP/2 200`. A `403` means a server rule is refusing it. A `404` after you fixed the rule means the file never reached the server, so go back to the build output. Also fetch `/.env` after changing a deny rule; it should still be refused.

## Next step

Run `npx aiscan-cli yoursite.com` and check P1 to P4 against the fix you applied. The same folder serves the [API catalog](https://aiscan.site/blog/publish-api-catalog-rfc-9727) too, and every other guide for these files is listed on [our guides page](https://aiscan.site/guides).