# How to Write an llms.txt File (Format, Template, and Maintenance)

> A step-by-step guide to writing a useful llms.txt: the exact format, a copy-paste template, what to put under ## Optional, how to validate it, and how to keep it from rotting.

Guide: The agent-ready web — part 2
Published: 2026-07-26 · Updated: 2026-07-26 · 1301 words
Canonical: https://changegamer.ai/articles/how-to-write-an-llms-txt-file
JSON: https://changegamer.ai/api/articles/how-to-write-an-llms-txt-file.json
Pillar: https://changegamer.ai/articles/agent-ready-website.md

## In short

- The format is four elements in order: an H1 name, an optional blockquote summary, optional prose, then H2 sections of annotated links. Nothing else is defined.
- Curate ruthlessly. A model choosing what to fetch benefits from forty good links, not four thousand — a dumped sitemap is the most common way to make the file worthless.
- Point every link at a Markdown variant where you have one. The file exists to reduce token cost; linking to JavaScript-heavy HTML defeats the purpose.
- Generate it from the same source that generates your pages. A hand-maintained llms.txt drifts within weeks and then lies about your site.
- Put changelogs, archives and secondary material under `## Optional` — agents under context pressure are permitted to skip that whole section.

---

An llms.txt file takes about an hour to write and most of them are wrong in the same two ways: they are dumped sitemaps, or they are hand-maintained and out of date. This is the process that avoids both.

For where this file sits in the wider picture, see the pillar: [the agent-ready website](/articles/agent-ready-website). For how it differs from the other root-level files, see [llms.txt vs robots.txt vs sitemap.xml](/articles/llms-txt-vs-robots-txt-vs-sitemap).

## The format, exactly

The convention — proposed by Jeremy Howard (Answer.AI) on 3 September 2024 — defines four elements, in this order. Only the first is required.

1. **H1** — the project or site name, nothing else: `# Example Corp`
2. **Blockquote** — one paragraph of context: `> What this site is.`
3. **Free-form prose** — paragraphs or bullets, no sub-headings.
4. **H2 sections** — repeatable named link lists.

Inside a section, every line is a link with an optional prose note:

```
- [Display name](https://example.com/page.md): what this page is for
```

One section name is special. `## Optional` marks links agents **may skip entirely** when they are short on context. Everything else is treated as core.

Serve the file at exactly `/llms.txt` with `Content-Type: text/plain; charset=utf-8`. Full details, including the agent-side consumption order, are in [the llms.txt convention explained](/resources/llms-txt-explained).

## Step 1 — Decide what the file is for

Write the answer to this in one sentence before writing any links: *what task is an agent trying to complete when it lands on my site?* The honest answer changes the file completely.

- **Documentation site:** the agent is trying to use your API correctly. Lead with quickstart, auth, and the reference — not with your blog.
- **Publisher:** the agent is trying to answer a question with a citable source. Lead with your evergreen explainers and your topic hubs, not with today's news (as of July 2026).
- **SaaS product:** the agent is trying to decide whether you fit a requirement. Lead with what the product does, pricing, and integration limits.
- **Reference corpus:** the agent wants a specific fact. Lead with the index and the machine formats.

## Step 2 — Choose sections that map to intent

Sections are the only structure a model gets, so name them for jobs, not for your internal org chart. Patterns that work:

```
## Start here          → orientation, 1–3 links
## Guides              → task-shaped how-tos
## Reference           → stable, lookup-shaped material
## API                 → machine entry points (JSON, OpenAPI, MCP)
## Policy              → terms, licensing, access, pricing
## Optional            → changelog, archive, secondary material
```

Resist the urge to mirror your navigation. Navigation is optimised for browsing; this file is optimised for a single decision — *which one URL do I fetch next?*

## Step 3 — Write the annotations, because they carry the file

The prose note after each link is what a model uses to choose. Two rules:

- **Say what the page contains, not what it is called.** `: installation, authentication, and a first working request` beats `: our quickstart guide`.
- **Be specific enough to be exclusionary.** A good note lets the model rule the page *out* as fast as it rules it in. `: pricing tiers, per-seat limits, and enterprise terms` tells a model that a question about rate limits is answered elsewhere.

Bad and good, same page:

```
- [Docs](https://example.com/docs): our documentation
- [API reference](https://example.com/docs/api.md): every endpoint with request and
  response shapes, error codes, and rate-limit headers
```

## Step 4 — Point at the cheapest representation you have

Link Markdown variants where they exist. If your pages are heavy HTML with navigation, cookie banners and script tags, every link in your llms.txt is an expensive fetch and the file has failed at its one job. Shipping Markdown twins is a small change with an outsized effect — see [serving Markdown variants to AI agents](/articles/serving-markdown-variants-to-ai-agents).

Where a JSON endpoint answers the question better than a page, link the endpoint. An llms.txt section listing `/api/*` entry points, each with a one-line note about what it returns, saves an agent from scraping HTML to reconstruct data you already publish structurally — see [JSON API design for agents](/articles/json-api-design-for-agents).

## Step 5 — Generate it, never hand-maintain it

This is the difference between a file that stays true and a file that lies. Derive llms.txt from whatever already knows your content — CMS query, content collection, or a data module — at build time, so a new or renamed page updates the index automatically.

A minimal generator in a static build:

```js
// pages/llms.txt.js — one source of truth, two rendered surfaces
export function GET() {
  const lines = [
    `# ${SITE.name}`, "",
    `> ${SITE.description}`, "",
  ];
  for (const section of SECTION_ORDER) {
    const pages = content.filter((p) => p.section === section && !p.draft);
    if (!pages.length) continue;
    lines.push(`## ${section}`, "");
    for (const p of pages) {
      lines.push(`- [${p.title}](${SITE.url}/${p.slug}.md): ${p.summary}`);
    }
    lines.push("");
  }
  return new Response(lines.join("\n"),
    { headers: { "Content-Type": "text/plain; charset=utf-8" } });
}
```

If your stack cannot generate it, the fallback is a scheduled job that regenerates the file and opens a pull request when it changes — anything that removes the human from the loop.

## Step 6 — Validate

There is no official validator, so check the things that actually break:

```bash
# 1. Served as plain text at the exact path
curl -sI https://example.com/llms.txt | head -3

# 2. Exactly one H1, at the top
curl -s https://example.com/llms.txt | grep -c "^# "

# 3. Every link resolves (and none redirects into a chain)
curl -s https://example.com/llms.txt \
  | grep -o "https://[^)]*" | sort -u \
  | while read -r u; do printf "%s %s\n" "$(curl -so /dev/null -w "%{http_code}" "$u")" "$u"; done \
  | grep -v "^200"

# 4. A crawler UA gets the same file a browser does
curl -s -A "GPTBot/1.2" https://example.com/llms.txt | head -5
```

Add step 3 to CI. Dead links in your machine index are worse than dead links in your footer, because an agent will not go looking for the working URL.

## A copy-paste template

```
# Acme Analytics

> Product analytics API and SDKs. Event ingestion, funnels, retention,
> and a query API. Self-serve from €0; SOC 2 Type II.

Prefer the .md variants linked below — they are the same content without
navigation or scripts. Structured data is at /api/openapi.json.

## Start here

- [What Acme does](https://acme.com/product.md): capabilities, limits, and what it is not
- [Quickstart](https://acme.com/docs/quickstart.md): install, authenticate, send a first event

## Guides

- [Event schema design](https://acme.com/docs/events.md): naming, properties, versioning rules
- [Backfills](https://acme.com/docs/backfill.md): importing historical events, dedup semantics

## Reference

- [Query API](https://acme.com/docs/query.md): endpoints, filters, pagination, rate limits
- [SDK matrix](https://acme.com/docs/sdks.md): languages, versions, supported features

## API

- [openapi.json](https://acme.com/api/openapi.json): OpenAPI 3.1 description of every endpoint
- [status.json](https://acme.com/api/status.json): current availability and incident history

## Policy

- [Pricing](https://acme.com/pricing.md): plans, usage limits, overage rates
- [Terms and licensing](https://acme.com/terms.md): permitted use, AI training terms

## Optional

- [Changelog](https://acme.com/changelog.md): releases, newest first
- [Blog archive](https://acme.com/blog.md): posts index
```

## Maintenance: the three things that rot

- **Descriptions.** Pages change more often than link text. If the file is generated from the same summaries your pages use, this fixes itself.
- **Section membership.** Material that becomes secondary should move to `## Optional`, not stay in `## Guides` forever.
- **Link targets.** Renaming a URL without a redirect breaks your machine index silently. Treat llms.txt as a public API surface: stable, tested, versioned with your site.

One more discipline, borrowed from how this site publishes: never gate the index. If a paid tier exists, the index may say a resource is premium and how to buy it, but the index itself and your terms page must stay free — otherwise an agent that hits your paywall cannot discover how to pay. The onboarding contract is written out in [getting started for agents](/resources/getting-started).

## Frequently asked questions

### How long should an llms.txt file be?

Short enough that a model can read the whole thing before deciding what to fetch — for most sites that is 20 to 100 links. If your file is hundreds of kilobytes you have built a sitemap in Markdown, which is a different file with a different job.

### Should the links point to .md files or HTML pages?

Markdown variants where they exist, HTML otherwise. The convention explicitly favours clean Markdown link targets because the point of the file is cheap consumption. If you cannot serve Markdown yet, link the HTML — an accurate index of HTML pages still beats no index.

### Do I need llms-full.txt as well?

Only if your corpus is small enough to inline sensibly and you expect agents to want all of it. It is widely adopted practice rather than part of the original proposal. If you publish one, keep it generated — never hand-assembled — and be deliberate about whether paid content appears in it.

### Will an llms.txt file improve my rankings?

There is no evidence that it does, and it is not a ranking factor in any published sense. It helps agents and tools that look for it. Publish it because it is an hour of work with no downside, not because you expect traffic from it.


---

## The rest of this guide

- [The Agent-Ready Website: A Complete Guide to AI Visibility, Access Control and Monetization](https://changegamer.ai/articles/agent-ready-website.md): The full operator playbook for making a website work for AI agents and AI crawlers: be fetchable, be readable, be controllable, be payable — with a 30-day implementation plan.
- [llms.txt vs robots.txt vs sitemap.xml: Which File Does What](https://changegamer.ai/articles/llms-txt-vs-robots-txt-vs-sitemap.md): The three root-level files every agent-ready site publishes, what each one is actually for, and why publishing one does not substitute for the others.
- [Serving Markdown Variants to AI Agents: The Cheapest Win in AI Visibility](https://changegamer.ai/articles/serving-markdown-variants-to-ai-agents.md): How to publish a .md twin of every page — URL patterns, content negotiation, discovery headers, generation pitfalls — and why it cuts what an agent pays to read you.
- [How AI Search Engines Choose Sources (And What You Can Actually Influence)](https://changegamer.ai/articles/how-ai-search-engines-choose-sources.md): What is known, what is claimed and what is speculation about how ChatGPT, Perplexity and AI Overviews pick the pages they cite — and the short list of things a site owner can actually control.
- [Should You Block AI Crawlers? A Decision Framework by Business Model](https://changegamer.ai/articles/should-you-block-ai-crawlers.md): Blocking AI crawlers is four separate decisions, not one. A framework that maps each crawler class to what it costs and earns you, by business model, with the exact robots.txt for each answer.
- [What to Charge AI Crawlers: Pricing Models for Machine Buyers](https://changegamer.ai/articles/what-to-charge-ai-crawlers.md): Per-crawl, per-resource, corpus licence or subscription key — the four ways to price AI access, the arithmetic behind each, and why pricing before you have demand data is the standard mistake.
- [Implementing an HTTP 402 Paywall an Agent Can Actually Pay](https://changegamer.ai/articles/http-402-paywall-implementation.md): A working implementation guide for machine-payable content: the 402 response body, Link headers, key issuance and validation, caching rules, and the mistakes that make a 402 gate unpayable.
- [Structured Data for AI Agents: Which Schema.org Types Earn Their Keep](https://changegamer.ai/articles/structured-data-for-ai-agents.md): Most schema.org markup is invisible to machine readers. The types that are worth the effort for AI agents, how to emit them without drift, and what to build instead of more markup.
- [JSON API Design for AI Agents: Endpoints They Prefer Over Scraping](https://changegamer.ai/articles/json-api-design-for-agents.md): How to publish read-only JSON endpoints that agents choose over scraping your HTML: discovery index, stable shapes, freshness signals, bulk exports, and errors a machine can act on.
- [Measuring AI Agent Traffic: Server-Side Telemetry That Answers Real Questions](https://changegamer.ai/articles/measuring-ai-agent-traffic.md): Why client-side analytics miss AI agents entirely, the minimum row schema to log, the five queries worth running, and how to tell a real crawler from a spoofed user agent.
- [Licensing Content for AI Training: RSL, Terms, and Provenance](https://changegamer.ai/articles/licensing-content-for-ai-training.md): How to publish machine-readable licence terms for AI use — what RSL is, what it does and does not do, how it differs from robots.txt and Content Signals, and where provenance standards fit.
- [Running an MCP Server as a Distribution Channel for Your Content](https://changegamer.ai/articles/mcp-server-as-distribution-channel.md): Why a content site should expose an MCP server, which tools to ship, how discovery and authentication work, how to gate paid tools, and the honest limits of the channel.
- [Why AI Agents Can't Read Your Site: Twelve Failure Modes and How to Find Them](https://changegamer.ai/articles/why-ai-agents-cant-read-your-site.md): A diagnostic catalogue of the twelve reasons AI agents and crawlers fail on real sites — from silent WAF blocks to JS-only rendering — each with the command that detects it and the fix.

## Reference resources

- https://changegamer.ai/resources/llms-txt-explained.md
- https://changegamer.ai/resources/data-formats.md
- https://changegamer.ai/resources/getting-started.md

All guides: https://changegamer.ai/api/articles.json · Reference corpus: https://changegamer.ai/llms.txt