← All guides · The agent-ready web

Implementing an HTTP 402 Paywall an Agent Can Actually Pay

Part 7 of The agent-ready web · 1,262 words · published 2026-07-26 · updated 2026-07-26 · Markdown variant

A working implementation guide for machine-payable content: the 402 response body, Link headers, key issuance and validation, caching rules, and the mistakes that make a 402 gate unpayable.

Part of the The Agent-Ready Website: A Complete Guide to AI Visibility, Access Control and Monetization guide.

Most paywalls are unpayable by machines: they redirect to a marketing page, return 200 with a teaser, or require an email confirmation loop. This is how to build one that a competent agent can complete end to end, which is the money half of the agent-ready website.

The wire contract this page implements is documented in paying for access: the HTTP 402 flow; the rail comparison is agentic payment protocols.

The loop, in five steps

1. GET /resources/premium-thing            → 402 + JSON body + Link headers
2. (agent parses price, terms, payment_url)
3. buyer/agent completes checkout           → your webhook mints a key
4. GET /resources/premium-thing            → with: Authorization: Bearer cg_live_…
5. 200 OK + Cache-Control: no-store        → content

Everything below is detail on making each step machine-completable.

Step 1 — The 402 response

This body is your entire sales page for a machine buyer. Five things are mandatory:

{
  "error": "payment_required",
  "resource": "premium-thing",
  "title": "The Premium Thing",
  "description": "What this resource contains, in one sentence.",
  "outline": ["Section one", "Section two", "Section three"],
  "words": 2400,
  "updated": "2026-07-20",
  "price": 5,
  "currency": "EUR",
  "payment_url": "https://buy.example.com/premium-key",
  "how_to_pay": "Buy an access key at payment_url, then retry this request with header 'Authorization: Bearer <key>'.",
  "pricing": "https://example.com/api/pricing.json",
  "terms": "https://example.com/terms.md",
  "license": "https://example.com/license.xml",
  "free_alternatives": ["https://example.com/llms.txt", "https://example.com/resources/overview.md"]
}

Design notes that matter more than they look:

• how_to_pay is prose for a machine. Spell out the header name and the retry. An agent that has to infer your auth scheme will guess wrong.
• outline and words are the preview. Structure without substance: enough to evaluate, not enough to substitute.
• free_alternatives reduces hostility. An agent that hits a wall and is told what it can have stays on your site.
• Content type is JSON. Never HTML. An error page is not a payment interface.

Step 2 — Advertise terms in headers as well

Headers let a HEAD request learn the commercial terms with no body transfer:

HTTP/1.1 402 Payment Required
Content-Type: application/json; charset=utf-8
Cache-Control: no-store
Link: <https://buy.example.com/premium-key>; rel="payment",
      <https://example.com/api/payment.json>; rel="payment-manifest",
      <https://example.com/api/pricing.json>; rel="pricing",
      <https://example.com/license.xml>; rel="license"

Pair this with a machine-readable payment manifest at a stable URL describing your accepted methods and the retry loop, plus a pricing catalogue listing every tier. Those two documents are what let an agent decide without triggering a 402 first — see JSON API for agents for the shape.

Step 3 — Issue keys without a human in the loop

The payment provider's webhook is what turns a completed checkout into an entitlement:

// Webhook handler (Cloudflare Worker + KV shown; any KV/DB works)
export async function onPaymentCompleted(event: CheckoutEvent, env: Env) {
  // 1. Verify the webhook signature. Never mint on an unverified event.
  if (!(await verifySignature(event.raw, env.WEBHOOK_SECRET))) {
    return new Response("bad signature", { status: 400 });
  }
  // 2. Idempotency: the same event may arrive more than once.
  const existing = await env.KEYS.get(`session:${event.sessionId}`);
  if (existing) return json({ status: "duplicate", key: existing });

  // 3. Mint a prefixed, high-entropy key. Prefix = greppable in logs and
  //    in leaked-secret scanners.
  const key = `cg_live_${crypto.randomUUID().replace(/-/g, "")}`;
  const record = { tier: event.tier, created: new Date().toISOString() };
  await env.KEYS.put(`key:${key}`, JSON.stringify(record));
  await env.KEYS.put(`session:${event.sessionId}`, key);

  // 4. Deliver: return it in the response AND make it retrievable once
  //    at a URL bound to the session id, so an agent that lost the
  //    response body can still fetch it.
  return json({ status: "ok", key, tier: event.tier });
}

Three rules learned the hard way:

1. Idempotency is not optional. Payment webhooks retry. Without a session-keyed guard you will mint several keys for one purchase and lose the ability to reason about entitlements.
2. Log the tier, never the key. Telemetry that records minted keys turns your log store into a credential store.
3. Provide a retrieval path. An agent may complete checkout in a context where it cannot keep the response. A GET /key?session=… that returns the key for a completed session — and a clear "pending" state if the webhook has not landed yet — closes that gap.

Step 4 — Validate at the edge, in front of the content

Validation must happen before the asset is served, not inside a rendered page. In a Workers-style architecture:

export default {
  async fetch(req: Request, env: Env) {
    const url = new URL(req.url);
    const slug = premiumSlugFor(url.pathname);      // null for free paths
    if (!slug) return env.ASSETS.fetch(req);        // free content, untouched

    const key = bearer(req) ?? req.headers.get("x-api-key");
    const tier = key ? await tierOf(key, env) : null;

    if (tier && tierAllows(tier, minTierFor(slug))) {
      const res = await env.ASSETS.fetch(req);
      return noStore(res);                          // never shared-cache paid bodies
    }
    return build402(slug, env, { upgrade: tier !== null });
  },
};

Details that decide whether this is robust:

• Accept two header spellings. Authorization: Bearer <key> and X-API-Key: <key>. Agents differ; accepting both costs one line.
• One 402 builder, used everywhere. The HTTP gate, the tier-upgrade path and any tool-call interface (MCP, for instance) must return the same shape from the same function, or your documented contract and your behaviour will drift.
• Same shape for "no key" and "wrong tier". Only how_to_pay changes: buy, versus upgrade. One parser handles both.
• Never paywall index or policy files. Keep robots.txt, llms.txt, sitemaps, your licence and your terms outside the gate by construction — ideally by routing, so no future code change can accidentally include them. A payment loop whose terms are behind the paywall is a dead end.

Step 5 — Caching, or how to leak the product

The single most expensive mistake in this design is a cacheable paid response. Rules:

If your CDN applies blanket caching headers by path pattern, verify that paid paths are excluded — with an actual request, using a key, checking the response headers. Assume nothing about inherited config.

Testing it like an agent would

# 1. Wall present and machine-readable
curl -s -i https://example.com/resources/premium-thing.md | head -20

# 2. The advertised terms are reachable WITHOUT paying
curl -so /dev/null -w "%{http_code}\n" https://example.com/terms.md
curl -so /dev/null -w "%{http_code}\n" https://example.com/license.xml

# 3. A valid key unlocks it, and the response is not cacheable
curl -s -i -H "Authorization: Bearer $KEY" \
  https://example.com/resources/premium-thing.md | grep -i cache-control

# 4. An invalid key returns the SAME 402 shape, not a 500
curl -s -H "Authorization: Bearer nope" \
  https://example.com/resources/premium-thing.md | head -5

# 5. Free surfaces are untouched by the gate
for p in /robots.txt /llms.txt /sitemap.xml; do
  printf "%s " "$p"; curl -so /dev/null -w "%{http_code}\n" "https://example.com$p"
done

Make tests 2 and 5 permanent CI assertions. They encode the promise that discovery and terms are free, which is the one invariant that must survive every future refactor.

What this does not solve

A 402 gate prices access; it does not prevent redistribution, and it does not express a licence. Those are separate layers: RSL states terms, provenance standards make origin verifiable, and neither collects money — see licensing content for AI training. And a gate is only as good as its demand: if you have not measured which content agents want, you are guessing at both the wall and the price, which is the argument of what to charge AI crawlers and measuring AI agent traffic.

#402 #paywall #monetization #agents #implementation