← All guides · Selling to AI agents

Packaging a Corpus as a Product: Format, Schema, Versioning and Delivery

Part 9 of Selling to AI agents · 1,622 words · published 2026-08-04 · updated 2026-08-04 · Markdown variant

The packaging decisions behind selling a content corpus as a dataset product — export format, the free-sample/gated-full split, a per-record metadata schema, a corpus version number, and which of three delivery mechanisms to use — grounded in ChangeGamer's own three real export formats.

Part of the How to Sell to AI Agents: The Complete Guide to Machine Buyers guide.

Once a corpus has a license and a price, someone still has to decide what file format ships, how the free preview differs from the paid deliverable, how a buyer validates what they received, how version changes get signaled, and how the file reaches them. Those are packaging decisions, not legal or pricing ones — this article sits inside how to sell to AI agents, the pillar for this cluster, and covers the mechanics of shipping a corpus once the legal and pricing decisions are made elsewhere. It does not cover licensing terms (see licensing content for AI training) or tier-pricing design (see pricing tiers for machine buyers).

Pick a format for what the buyer will actually do with it

The right export format follows from the buyer's pipeline, not from whichever is easiest to generate, and one corpus product often needs more than one shape. ChangeGamer ships three:

• NDJSON (newline-delimited JSON) — /api/corpus.jsonl and its gated counterpart /api/corpus.full.jsonl. One JSON object per line, no enclosing array, so a consumer can stream it line-by-line straight into a vector-store indexer or a training job without loading the whole file into memory first. This is the standard shape for RAG ingestion and bulk fine-tuning data.
• A full-text dump — ChangeGamer's version of this is /llms-full.txt, the whole free corpus concatenated into one document (covered in full in llms.txt vs. robots.txt vs. sitemap.xml). It is the right shape for a single context-window load — "give me everything in one fetch" — and the wrong shape for streaming ingestion or per-record processing, since there is no record boundary a parser can rely on beyond markdown headings.
• A chat-format supervised-fine-tuning (SFT) export — /api/finetune-sample.jsonl, one {"messages": [{"role": "system", ...}, {"role": "user", ...}, {"role": "assistant", ...}]} object per line, the shape most fine-tuning APIs expect natively. It is a different transformation of the same data, not a reformatted NDJSON line: ChangeGamer derives up to three training-example types per resource (an "explain" example, a one-sentence "summarize" example, and a "key facts" example built from that resource's own ## Key facts bullets).

Worth naming precisely because it is easy to get backwards: /api/finetune-sample.jsonl is always free and excludes every premium resource from all three task types — it is not a gated Corpus-tier deliverable, and it never becomes "the paid version" at any tier. It exists to demonstrate the corpus as training data using only what is already free, so a buyer can evaluate the shape of the paid product without a key.

The sample/full split is a field-level contract, not a shorter file

A free sample and a gated full export should have the identical record set and shape, differing only in which field values are populated — not a shorter file with fewer records. ChangeGamer's two NDJSON exports demonstrate the pattern exactly:

The access object is the load-bearing detail: an inline pointer — reason, unlock_with_key, key_offer, checkout_url, offer_catalog, payment_manifest — so an agent hitting a null-text record knows exactly how to unlock it without a second discovery hop. sources (citation URLs from each resource's own "Verified sources" section) is deliberately populated on both files, even on stubbed premium records, because it is metadata about the resource, not paywalled content — a useful line for any seller deciding what belongs in a free sample: content is gated, metadata about the content generally is not.

A schema per record, published, not implied

A buyer should be able to validate a dataset export against a published schema instead of inferring field shapes from whatever sample they happen to have. ChangeGamer publishes /api/corpus.schema.json, a JSON Schema (draft 2020-12) covering both record types in its NDJSON exports: a meta record (always the first line — site, url, description, corpus_version, count, note, license) and a resource record (every subsequent line — record, url, slug, title, description, category, tags, updated, premium, text, plus the conditional access and always-present sources fields above). The schema's category enum is derived live from the actual resource data at build time rather than hand-maintained — a schema that can silently drift from the data it describes is worse than no schema, since it fails the one job (catching a malformed export) that justifies publishing it.

Versioning: one number for content, a different one for the server

A corpus needs its own version number, independent of any API-server or MCP-manifest version, because the two answer different questions for a buyer. ChangeGamer maintains CORPUS_VERSION as a manually-bumped semver string, currently 1.12.11: MAJOR for a breaking schema change (a stats or record field renamed or removed), MINOR for a resource added or removed, or a new additive field, PATCH for a content-only edit with no count change. That number is stamped into every corpus export's header and into /api/stats.json's corpus_version field, and is explicitly documented as something that must not track the MCP server manifest's own version field — the corpus number tracks content and schema, the server number tracks server-implementation changes, and conflating them would fire a buyer's re-pull check on changes unrelated to the data they licensed. If your dataset product and API server ship from one repository, resist giving them one version number: a buyer polling for freshness wants to know one thing, did the data change.

Freshness is a version comparison, not a file timestamp

Tell a buyer to compare corpus version numbers, not to eyeball a last-modified date, because a version bump is a deliberate signal and a file timestamp is not. ChangeGamer's per-record updated date is scoped narrowly — bumped only on a genuinely re-verified content delta, not a structural or formatting change — so it is a record-level signal, while corpus_version is the corpus-level one: every export header carries the version it was built from, and a keyed buyer compares that stamped number against the live value in /api/stats.json (also shown on the public /changelog page) to decide whether to re-pull.

What "re-pull" means also depends on which tier a buyer is on — a genuine point-in-time-versus-standing-subscription distinction, not a technicality. Per access and pricing for crawlers, ChangeGamer's one-time Corpus and Enterprise licenses deliver a snapshot — the license grant covers the version delivered at purchase, and getting anything shipped afterward means fetching /api/corpus.full.jsonl again with the same key. The annual Corpus Updates subscription instead re-sends the full corpus file at every renewal, always reflecting whatever is current at that date, with re-delivery performed manually by the operator rather than automatically. Choosing between a one-time license and a subscription is, in large part, choosing who is responsible for noticing a re-pull is due — the buyer, or the seller's own renewal process.

Three delivery mechanisms, three different tradeoffs

Offer more than one delivery mechanism: "pull," "push," and "agent-native tool call" solve for different combinations of latency and trust. ChangeGamer runs all three for the same corpus file, per access and pricing for crawlers:

• Pull endpoint — GET /api/corpus.full.jsonl with Authorization: Bearer <key>. Immediate: a keyed buyer fetches the current full corpus the moment they want it, no wait, no human in the loop. The tradeoff is that it assumes machine-to-machine trust already exists via the key itself, and requires an HTTP client capable of the fetch.
• Emailed file — the full llms-full.txt corpus file sent to the checkout email address within one business day of purchase, and re-sent at each annual renewal. Slower by design: it routes through a human-verified email address rather than an API key alone, which matters for a buyer who wants a durable, addressable record of what they were sent and when, independent of whether their key is later lost or revoked.
• MCP tool call — get_full_corpus(api_key), exposed on ChangeGamer's /mcp server. Same immediate latency as the pull endpoint, but reachable as a typed tool call from inside an agent's own MCP client rather than a raw HTTP fetch its operator has to wire up separately — the difference here is discoverability inside an agent's existing toolset, not trust or speed.

None of the three replaces the others: the pull endpoint and the MCP tool serve an agent that already has a key and wants current data now; the emailed file serves the human on the other end of the purchase who wants a dated, addressable copy independent of key lifecycle. A seller who only builds whichever is easiest to implement is choosing a trust or latency profile by accident, not by design.

What this means for packaging your own corpus

Decide the export format from the buyer's pipeline first — NDJSON for streaming ingestion, a chat-format JSONL for fine-tuning, a full-text dump only for one-shot context loads — and ship more than one if buyers need more than one job done. If you run a free sample next to a paid full export, make the difference specific withheld field values on an identical record set, not a shorter file, with an actionable unlock pointer left in place of each withheld value. Publish a schema for the record shape, derived from live data so it cannot drift. Version corpus content independently from server or manifest version. And offer more than one delivery mechanism if your buyers mix machines that want to pull now with humans who want a dated, addressable copy.

#corpus #packaging #ndjson #versioning #delivery #monetization #agents