WordPress

WordPress

Make a WordPress site agent-ready: robots.txt, llms.txt, sitemap, and AI bot rules.

WordPress powers more than 40% of the web, and most of the AIScan checks map to either a plugin setting or a small functions.php snippet. The Content dimension gets a 1.2× weight on WordPress because content is what readers (and agents) come for.

robots.txt

WordPress generates a virtual /robots.txt by default — but it's empty beyond Disallow: /wp-admin/. Yoast SEO and Rank Math both expose an editor that lets you append rules without dropping a real file.

Hand-rolled (functions.php)

add_filter('robots_txt', function ($output, $public) {
  if (!$public) return $output;
  $extra = <<<TXT

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

Sitemap: https://example.com/sitemap_index.xml
TXT;
  return $output . $extra;
}, 10, 2);

Sitemap

WordPress 5.5+ ships a core sitemap at /wp-sitemap.xml. If you use Yoast or Rank Math, prefer their /sitemap_index.xml — they include more URLs and lastmod dates. Either way, reference the canonical sitemap from robots.txt.

llms.txt

Three approaches:

  • Drop a static file into your theme's root via FTP and route it with an .htaccess rewrite.
  • Use a plugin (search "llms.txt" in the plugin directory — several exist).
  • Add a template_redirect hook that intercepts /llms.txt and returns the markdown.

Structured HTML & JSON-LD

Yoast and Rank Math both emit schema.org JSON-LD automatically (Organization, WebSite, Article). Validate at schema.org validator. Most themes already ship a single <h1>, but classic themes sometimes use multiple — check with your theme author.

Markdown negotiation

WordPress doesn't ship a markdown renderer. The pragmatic path is to expose a sidecar URL per post via a plugin or a tiny rest_api_init route that returns the post body rendered as Markdown when the request includes Accept: text/markdown. Until then, this check sits at fail — which is fine; llms.txt covers most of the value at a tenth of the work.