LLMs.txt

/llms.txt not found

Your site has no /llms.txt file. This is the most impactful AX file you can create: it gives AI agents a structured, human-readable summary of your site in Markdown format.

Create a llms.txt file in your site root following the llmstxt.org specification:

# Your Site Name

> A brief summary of what your site does, who it's for, and what AI agents can find here.

## Sections

- [About](https://your-site.com/about): Company overview
- [API](https://your-site.com/api): REST API documentation
- [Docs](https://your-site.com/docs): Full documentation

## API

- [Users](https://api.your-site.com/users): GET /users
- [Products](https://api.your-site.com/products): GET /products

## Optional

- [Full LLM docs](https://your-site.com/llms-full.txt): Extended version

Wrong Content-Type

Serve /llms.txt with Content-Type: text/plain or text/markdown. Some agents reject text/html responses on this path because the file is supposed to be plain Markdown, not a rendered HTML preview.

# Nginx
location = /llms.txt {
  default_type text/plain;
  charset utf-8;
}

# Vercel (vercel.json)
{
  "headers": [{
    "source": "/llms.txt",
    "headers": [{
      "key": "Content-Type",
      "value": "text/plain; charset=utf-8"
    }]
  }]
}

Missing H1 heading

The llmstxt.org spec requires the first line to be an H1 heading (# Your Site Name). This tells AI agents what entity the file describes.

# Your Site Name

Make sure the # is the very first character of the file, followed by a space and your site or project name.

Missing blockquote description

After the H1 heading, the spec expects a blockquote (> ...) containing a brief summary. This is the first thing an AI agent reads to understand your site.

# Your Site Name

> A one-line description of what your site offers and who it's for.

Keep it concise (1-2 sentences). Think of it as a meta description optimized for AI agents rather than search engines.

Missing section headings

Your llms.txt has no ## section headings. These help AI agents navigate the content and understand its structure.

Common sections include:

• ## Sections: links to main site pages
• ## API: API endpoints and documentation
• ## Documentation: technical docs links
• ## Optional: extended resources and full LLM docs

No Markdown links

Your llms.txt contains no Markdown links ([text](url)). Links are essential: they allow AI agents to navigate from the summary to specific pages.

- [Home](https://your-site.com/): Overview and navigation
- [API Docs](https://your-site.com/api): Full API reference
- [Contact](https://your-site.com/contact): Get in touch

Use absolute URLs (including https://) so agents can resolve them from any context.

Minimal content

Your llms.txt has fewer than 100 characters. While there's no strict length requirement, very short files don't give AI agents enough context to understand your site.

A good llms.txt typically includes:

• H1 with your name/brand
• Blockquote summary (1-2 sentences)
• Key information (title, location, contact, etc.)
• 2-3 sections with descriptive links
• Total: 500-3,000 characters is a good range

Missing /llms-full.txt

The /llms-full.txt file is optional but recommended. It's an extended version of your llms.txt with comprehensive detail: full API documentation, complete technology lists, detailed descriptions, etc.

Think of it this way: llms.txt is the summary, llms-full.txt is the full documentation. An AI agent can choose which version to read based on its context window and needs.

/llms.txt is too large

Your file is over 50 KB. An index that size competes for context with the content it points at: the agent spends its budget reading the map instead of the territory, and past a point it simply truncates.

llms.txt is a list of links with one line of orientation each. If yours has grown into prose, that prose belongs in llms-full.txt or on the pages themselves, both of which the agent can fetch once it knows what it wants.

No rel="describedby" link to llms.txt

An agent rarely arrives at your homepage. It lands three levels deep on the page that answered a search, and from there it has no way to know an index exists, unless the page says so.

llms.txt v2 uses the describedby relation for exactly this. Emit it as a header or a link tag; either is found.

Link: </llms.txt>; rel="describedby"; type="text/plain"

<link rel="describedby" href="/llms.txt" type="text/plain">

A page may point at a section-scoped file rather than the root one, /docs/llms.txt for everything under /docs, which is the other half of why the relation exists.

Duplicate links

The same URL is listed more than once. An agent working through the index has no cheap way to know it has already read a page, so a duplicate is a fetch and a context window paid for twice.

Duplicates usually come from generating the file per section and concatenating: a page that belongs to two sections lands in both. List each URL once, under the section where a reader would look for it first.

Broken links

A sample of the URLs in your llms.txt does not resolve. This is the failure this file is most prone to: it is written once, by hand, and never revalidated, while the site keeps moving underneath it.

The cost is worse than a 404 in a browser. A person who hits one goes back and looks around; an agent reading an index of dead links has spent its budget on nothing and cannot tell a moved page from a deleted one.

Links that redirect

Some sampled URLs answer with a 3xx rather than the page. Nothing is broken, but every agent that follows the index pays an extra round trip for every one of them, on every visit.

Point the index at the final URL. Redirects here are a record of a move that was made everywhere except in this file, so they are also a reliable early warning for the broken links above.

No per-page Markdown mirror

The audit tried /index.md and /index.html.md and found no Markdown version of the homepage. llms.txt v2 documents appending .md to any URL for its Markdown representation.

The mirrors matter more than the index does. The index tells an agent which pages exist; the mirrors are what make reading them cheap: typically around a fifth of the tokens the HTML costs, with none of the navigation, scripts and markup the agent pays to receive and then discards.