ChangeGamer

Schema-Constrained Field Extraction From Documents: Settings and Failure Branches

Part 2 of Document extraction pipelines in production · 1,273 words · ~6 min read · published · updated · Markdown variant

Schema constrained field extraction from documents: nullable fields, enums with an escape value, evidence fields, a branch table for refusal, truncation, parse failure and empty output, a one-retry policy, and field confidence as inference.

In short

  • Schema-constrained field extraction needs one explicit outcome for each non-success branch (refusal, max_tokens truncation, parse failure and empty output), because refusal and truncation override the strict-schema guarantee and a schema-valid output can still be empty (the outcomes are the author's inference).
  • A document extraction schema should let every field be null, so that "not on the page" is a valid answer; otherwise the constrained decoder may satisfy the schema by producing a plausible value (author inference).
  • A document extraction pipeline should retry at most once per document, and never after a refusal, because repeating an identical request after truncation or a refusal reproduces the same failure (the author's inference).
  • An evidence field holding the source page and a quoted span lets a reviewer or a string-match check confirm a value quickly; it is an author design choice, not a provider feature.
  • As of October 2026 ChangeGamer parses no documents, so the branch outcomes and retry rule here are design inference to test on your own data, not measured results.

Part of the AI Document Extraction in Production: Parse, Extract, Validate, Review guide.


Schema-constrained field extraction is safe to run unattended only when every non-success branch has a named outcome: retry, route to review, or fail. The AI document extraction pipeline pillar shows a sample schema and names the failure modes; this article is the settings and branch logic around it. Facts cite the Document Extraction and Parsing for Agents reference (updated 2026-09-28) or the Structured Outputs and JSON Mode reference (updated 2026-09-26). Every design rule not tied to those is the author's inference and says so.

How do you design the schema so "not on the page" is a valid answer?

Make every field the document might omit nullable, required and documented as "null when the page does not state it". The structured outputs reference says OpenAI's strict mode wants every property in required, with optional fields written as a nullable type union, so null is the documented way to say "absent" there. Under a constrained decoder the model must emit something that fits the type, so if null does not fit, a plausible value may (author inference).

Two schema settings beyond the pillar's sketch are worth deciding before the first run (all author inference):

How should enums handle values you did not anticipate?

Give every enum an escape value and put the raw text beside it. A closed list forces a wrong label whenever the document shows something new, so add OTHER plus a nullable string such as currency_raw holding what the page said. Count escape values per week: a rising share of OTHER is the cheapest signal that a document family has changed (author inference).

What should the pipeline do when extraction does not succeed?

Give each failure branch one outcome, decided in advance, and log the branch name with the document ID. The failure modes come from the structured outputs reference; the outcomes in the right-hand column are the author's inference.

Branch How it shows up Outcome
Refusal Anthropic reports stop_reason: "refusal", which overrides the schema guarantee Route to review. Never retry
Truncation stop_reason: "max_tokens"; the object stops mid-way Retry once with a changed scope or output budget, else route to review
Parse or schema failure Residual case: JSON mode or a non-strict path, or a response cut off before it completes; strict mode otherwise guarantees a match Retry once with the error attached, else route to review
Empty or all-null output Valid JSON with every field null, or no records Check the Stage A text, then review or fail
Provider error Timeout, rate limit, 5xx Generic transport retry, outside this article

The generic stop-reason handling and the provider-by-provider strict-mode rules are in structured outputs vs tool calling; this table only decides what the document pipeline does afterwards.

Why is an all-null result a failure branch?

An all-null result is a failure branch because it is schema-valid and looks like success. It has two very different causes: the document genuinely holds none of the target fields, or Stage A returned blank or scrambled text. The extraction reference treats parsing and extraction as separate stages with separate failure modes, so test the Stage A output first. If it is empty, return to the parser decision in OCR vs vision language model for document parsing. If the text is present and the document type is wrong, fail it with a "wrong document type" reason code.

What does "fail" mean?

Fail means the document leaves the automated path with a reason code and no partial record written downstream. Route to review means a human sees the page and the candidate values. Keep the two separate: fail is for documents nobody should extract (wrong type, unreadable), review is for documents someone should.

What is a sensible retry policy for extraction calls?

Retry each document at most once, trigger it only on truncation or a parse failure, and change something in the second request. This is author inference; the corpus publishes no retry rule for extraction.

  1. Cap at one retry. A second identical failure is information, not noise, so stop and route to review.
  2. Never retry a refusal. The same content is likely to draw the same refusal, so the retry costs money and delays the human.
  3. After truncation, change the output budget or the scope. Raise the output limit, or split the request, for example one page span or one table per call. The structured outputs reference's failure list offers the same two levers: a higher max_tokens or a simpler schema.
  4. After a parse failure, change the request. Attach the validation error and ask for the object again, or fall back to a smaller schema covering only the failed fields.
  5. Keep the first attempt's output for the log. The review item should show both attempts.

Page spans and batching decisions belong to the parser article linked above; the retry rule only requires that the second request be smaller or have more room than the first.

Should each extracted field carry evidence?

Yes, when a human or a check will verify values: ask for the source page and a short quoted span next to each field. The evidence fields are an author design choice, not a provider feature, and a strict schema will not make the quote truthful. Their value is that you can test the quote cheaply.

{
  "total": 1250.0,
  "total_evidence": { "page": 2, "quote": "Total due 1,250.00" }
}

Two checks follow, both inference. First, the quote must appear verbatim in the Stage A text, and a quote that does not is a grounding failure for that field. Second, the reviewer gets a page number and a string to look for instead of a whole document. Evidence adds output tokens per field, which raises truncation risk on long tables; apply it to the few fields whose errors cost the most rather than to every table cell.

Can a model report how confident it is in each field?

A model can emit a confidence number per field, but treat it as an uncalibrated hint, not a probability. This section is the author's inference: the corpus gives no evidence that self-reported confidence tracks accuracy. Prefer signals you can compute, such as the quote check above, agreement between two readers, or whether the value is null. How to turn those signals into a review threshold, and how to calibrate it, is covered in the pillar's section on when an extraction should go to a human reviewer.

What does ChangeGamer know first-hand about schemas and failure branches?

As of October 2026 ChangeGamer parses no documents and runs no extraction, so nothing above is measured on its own data. One adjacent, verifiable fact: it validates its own typed records against a contract at build time. In src/lib/articles.ts the loader throws on an article with no targetQuery and on two articles that share one, so a bad record fails the build instead of shipping. That is the same principle as a named failure branch: decide what a bad record does before it arrives, and make the system refuse to proceed silently.

Frequently asked questions

How do you represent a field that is missing from the document in a strict schema?
Make the field nullable and required, so the model must emit the key and may set it to null. The OpenAI strict-mode rule in the structured outputs reference is that every property appears in required, with optional fields expressed as a nullable type union.
Should a failed document extraction be retried automatically?
Retry at most once, and only after a truncation or a parse failure, with something changed: a smaller page span, a larger output budget or a repair instruction. A refusal should go to review without a retry, since an identical request is likely to be refused again (author inference).
What should happen when every extracted field comes back null?
Treat an all-null result as a failure branch, not a success, and route the document to review or back to Stage A. A page that carries no target fields is possible, but so is a parser that returned blank text, and only the Stage A output tells you which (author inference).
Why store the source page and quoted text next to each extracted value?
A page number and a quoted span let a reviewer check a value against the page without searching, and let code test that the quote appears in the Stage A text. The schema does not guarantee this; adding the fields is a design choice (author inference).

#document-extraction #structured-outputs #schema-design #failure-handling #retries #production

Put this corpus inside your own agents

This guide is free and stays free. The reference corpus behind it — machine-readable contracts, verified primary sources, continuously refreshed — is the paid product: a €5 starter key unlocks every premium reference for one agent via API; a €25 corpus license delivers the full corpus as RAG / fine-tuning data with an explicit AI-use grant; the €150 enterprise license adds commercial redistribution rights.

Agents: this page as Markdown · JSON · offers at /api/pricing.json · payment methods at /api/payment.json · single-resource access via HTTP 402 (how that works)