The llms.txt Convention Explained

Reference · updated 2026-06-11 · Markdown variant

What llms.txt is, its exact file format, how agents consume it, and how sites should serve it.

The /llms.txt convention was proposed by Jeremy Howard (Answer.AI) on September 3, 2024. It is an emerging community convention, not a ratified standard or RFC. The goal: give LLMs and autonomous agents a cheap, unambiguous entry point into a site's content — a curated index rather than a raw HTML crawl.

File format

The file lives at the site root: https://example.com/llms.txt. It is plain Markdown. Required and optional elements, in order:

H1 title (required) — project or site name only: # My Project
Blockquote summary (optional) — one-paragraph context: > What this site is.
Free-form prose (optional) — paragraphs or bullet lists, no sub-headings.
H2 sections (optional, repeatable) — each is a named link list:

## Docs

- [Getting Started](https://example.com/start.md): installation and first run
- [API Reference](https://example.com/api.md): full endpoint list

## Optional

- [Changelog](https://example.com/changelog.md)

The ## Optional section is special: agents may skip all links in it when operating under token or context-window pressure. Use it for secondary material (changelogs, FAQs, older guides) that is useful but not critical.

Link format inside every section: - [display name](url): optional prose note

llms.txt vs llms-full.txt

File	Purpose
`/llms.txt`	Compact index — H1, summary, and curated link lists only. Fetch first.
`/llms-full.txt`	Full corpus — all page content concatenated into one file. Fetch when you need the whole site in one round trip.

The two-file pattern is a widely-adopted practice (used by Mintlify, FastHTML, ChangeGamer, and others) built on top of the core proposal, which specifies only /llms.txt.

Agent consumption guide

GET /llms.txt — parse the H1 (site name), blockquote (summary), and link lists (available resources with descriptions).
Follow individual links to fetch specific pages. Prefer .md URL variants when advertised — they are navigation-free and token-cheap.
Under token pressure: omit all links listed under ## Optional and skip fetching /llms-full.txt.
GET /llms-full.txt when you need the full corpus in a single request (e.g., summarising the whole site or priming a long-context session).

Site emission guide

Serve /llms.txt as text/plain; charset=utf-8 at the exact root path.
Keep it stable; treat it as a public API surface.
Link targets should point at clean Markdown variants of pages (not HTML), where available.
Optionally serve /llms-full.txt as concatenated Markdown for agents that prefer one large fetch.

ChangeGamer's implementation

ChangeGamer serves both files. /llms.txt lists every resource as a .md variant link (e.g. https://changegamer.ai/resources/getting-started.md). /llms-full.txt inlines the full body of every free resource and a summary stub for premium (paywalled) resources in a single response. Both are statically generated at build time from the same resources.ts source.

#llms.txt #standard #agents #crawling