# Structured Outputs and JSON Mode: Provider Reference

> How to force schema-valid JSON from OpenAI, Anthropic, and Gemini — parameter names, strict-mode requirements, schema-subset limits, and self-hosted constrained decoding.

Category: Guide · Updated: 2026-07-02 · Tags: structured-outputs, json-mode, response-format, json-schema, constrained-decoding, agents
Canonical: https://changegamer.ai/resources/structured-outputs-and-json-mode
Variants: [HTML](https://changegamer.ai/resources/structured-outputs-and-json-mode) · [JSON](https://changegamer.ai/api/resources/structured-outputs-and-json-mode.json)
License: https://changegamer.ai/license.xml · Access: free

Structured outputs force an LLM to return valid JSON that conforms to a specific schema, without requiring tool calls. This is distinct from tool calling: the model's *final reply* is the schema-constrained object, not an intermediate function invocation. See /resources/reliable-tool-calling for the tool-calling primitive and how they interact.

## Key facts

- Structured outputs constrain the model's own final reply into a schema-valid object, which is a distinct mechanism from tool calling's intermediate function invocation.
- True schema-constrained decoding (`json_schema` / strict mode) physically prevents schema-violating tokens, whereas legacy JSON mode only asks the model for syntactically valid JSON with no schema enforced — which is why OpenAI now treats plain JSON mode as legacy.
- Each provider enforces and limits schemas differently: OpenAI's strict mode disallows default values and root-level `anyOf`, Anthropic's GA structured outputs support `anyOf`/`$ref` but not recursion, and Gemini splits into an OpenAPI-subset schema for older models versus a fuller JSON Schema for 2.5+.
- Self-hosted serving stacks push schema enforcement down to token sampling via grammars — vLLM and SGLang default to the XGrammar library for this, while llama.cpp relies on its own GBNF grammar format.
- Even under a strict schema guarantee, output can still come back non-compliant in practice if generation hits a token-limit truncation or a safety refusal — so the stop reason always needs checking.
- First-request latency can rise because of grammar compilation, which is mitigated by a warm-up call and avoiding frequent schema churn.
- Major SDKs from OpenAI, Anthropic, and Gemini all offer helpers that derive the schema directly from a Pydantic or Zod model and parse the result.

## Which mode to use

**Structured outputs / `json_schema`** — the provider's constrained decoder physically prevents the model from emitting tokens that violate your schema. Guarantees schema compliance given no refusal and no token-limit truncation.

**JSON mode / `json_object`** — the model is merely instructed to produce syntactically valid JSON; no schema is enforced, so it may omit fields, use wrong types, or add keys. OpenAI now labels this mode legacy — prefer `json_schema` with `strict: true`.

## Provider mechanisms

### OpenAI
Set `response_format: { type: "json_schema", json_schema: { name, schema, strict: true } }`. With `strict: true`, constrained decoding guarantees the output matches the schema. Requirements: every object must set `additionalProperties: false`; every property must appear in `required` (make optional fields a nullable type union). Supported on gpt-4o (2024-08-06+), GPT-4.1, GPT-5, and the o-series. Older snapshots support only the weaker `json_object` mode.

**Schema subset — not supported by strict mode:** `default` values (the call fails); root-level `anyOf` (use a nullable type union instead); numeric/string constraints like `minimum`/`pattern` (accepted but not enforced by the decoder); very deep nesting / very large property counts. `response_format` and `tools` are mutually exclusive in a single call.

### Anthropic (Claude)
Native structured outputs shipped in late 2025 and are now GA on current models — no beta header required. Set `output_config: { format: { type: "json_schema", schema } }`; the schema is compiled to a grammar and generation is constrained at decode time. Also supports `strict: true` on tool definitions for guaranteed tool-input validation. Notes: `anyOf`/`$ref`/`$defs` supported; numeric range and recursive schemas are not; per-request limits apply (caps on tool count, optional/union parameters, and grammar-compilation time — check the docs for current values); a schema change invalidates the grammar cache, adding first-request latency; `stop_reason: "refusal"` or `"max_tokens"` bypasses the guarantee, so always check `stop_reason`.

### Google Gemini
Set `responseMimeType: "application/json"` together with a schema. Two schema formats: **`responseSchema`** (an OpenAPI-based subset; no `anyOf`/`$ref`; Gemini 1.5/2.0) and **`responseJsonSchema`** (full JSON Schema with `anyOf`/`$ref`; Gemini 2.5+, accepts Pydantic/Zod schemas). Known limitation: in some 2.5 snapshots structured output fails when tool-call messages are in history; overly complex schemas can trigger a 400 — shorten property names, reduce enum size, or flatten nested arrays.

### Self-hosted / open models
Pass the schema at the serving layer, not in the prompt. vLLM and SGLang accept `guided_json` (JSON Schema), `guided_grammar` (EBNF/GBNF), or `guided_regex`. XGrammar (the default backend for vLLM/SGLang) compiles JSON Schema to a pushdown automaton with low per-token overhead; Outlines (dottxt-ai) uses FSM-based masking but does not support recursive schemas — use XGrammar or llguidance for those. llama.cpp uses GBNF grammars.

## Provider comparison

| Provider | Parameter | Strict guarantee? | anyOf | Defaults | Recursion |
|---|---|---|---|---|---|
| OpenAI (gpt-4o+, GPT-4.1+, GPT-5) | `response_format: json_schema, strict:true` | Yes | Root not allowed; use type unions | No (call fails) | No (depth cap) |
| OpenAI (legacy `json_object`) | `response_format: json_object` | Syntax only | N/A | N/A | N/A |
| Anthropic (recent Claude models) | `output_config.format: json_schema` | Yes | Yes | Yes | No |
| Gemini 1.5 / 2.0 | `responseMimeType` + `responseSchema` | Subset | No | N/A | No |
| Gemini 2.5+ | `responseMimeType` + `responseJsonSchema` | Full JSON Schema | Yes | Yes | Limited |
| vLLM / SGLang (XGrammar) | `guided_json` | Yes | Yes | N/A | Yes |
| llama.cpp | `grammar` (GBNF) | Yes | Via grammar | N/A | Yes |

## Common failure modes

- **API rejected (400)** — schema uses unsupported features (`default`, numeric ranges, recursive `$ref`). Strip unsupported keywords (most SDKs do this automatically).
- **Output truncated mid-object** — `stop_reason: "max_tokens"`; the schema cannot prevent an early stop. Always check the stop reason; raise `max_tokens` or simplify the schema.
- **Schema ignored despite no error** — you used `json_object` instead of `json_schema` strict.
- **Model refuses** — a safety refusal overrides the schema (`stop_reason: "refusal"`); treat it as a first-class error branch.
- **First request slow** — grammar compilation; warm up with a test call and avoid schema churn.

## SDK helpers

OpenAI `client.beta.chat.completions.parse(response_format=PydanticModel)` (Python) / `zodResponseFormat(schema, name)` (TS); Anthropic `messages.parse(output_format=PydanticModel)` (on `messages.create` itself the canonical parameter is `output_config.format` — the top-level `output_format` create-param is deprecated); Gemini (google-genai) accepts a Pydantic class directly as `response_schema`. These auto-derive the schema and parse the result.

## Verified sources

- Anthropic structured outputs docs: https://platform.claude.com/docs/en/build-with-claude/structured-outputs
- Google Gemini structured output docs: https://ai.google.dev/gemini-api/docs/structured-output
- OpenAI Structured Outputs guide: https://developers.openai.com/api/docs/guides/structured-outputs
- XGrammar paper (constrained generation, arXiv:2411.15100): https://arxiv.org/abs/2411.15100
- Outlines (dottxt-ai): https://github.com/dottxt-ai/outlines

---

## Related resources

- [Reliable Tool Calling and Structured Outputs](https://changegamer.ai/resources/reliable-tool-calling.md): How providers guarantee schema-valid tool calls and structured output — mechanisms, failure modes, and mitigations — for production agent builders.
- [Agent Cost and Latency Optimization](https://changegamer.ai/resources/agent-cost-latency-optimization.md): Practitioner reference for reducing the cost and latency of production AI agents: the compounding model, token-level levers (caching, pruning), request-level levers (Batch API, parallelism), model-level levers (routing, reasoning-effort controls), and architecture-level levers (step reduction, semantic caching, code offloading).
- [Agent Identity and Authentication](https://changegamer.ai/resources/agent-identity-authentication.md): How autonomous agents prove who they are and get authorized to act: workload identity vs. delegated authority, SPIFFE/SPIRE, cloud workload federation, OAuth token exchange, audience binding, and emerging standards — with practical guidance and verified sources.
- [Agent Memory and Context Management](https://changegamer.ai/resources/agent-memory-context.md): Architecture reference for agent memory: types (working, long-term, episodic, semantic, procedural), context-management techniques (summarization, RAG, sliding windows, prompt caching), storage substrates, and memory frameworks — with security notes and cross-links to related guides.

---

Index of all resources: https://changegamer.ai/llms.txt · Full corpus: https://changegamer.ai/llms-full.txt · Corpus data (NDJSON): https://changegamer.ai/api/corpus.jsonl · Offers: https://changegamer.ai/api/pricing.json