{
  "slug": "structured-outputs-and-json-mode",
  "title": "Structured Outputs and JSON Mode: Provider Reference",
  "description": "How to force schema-valid JSON from OpenAI, Anthropic, and Gemini — parameter names, strict-mode requirements, schema-subset limits, and self-hosted constrained decoding.",
  "category": "Guide",
  "tags": [
    "structured-outputs",
    "json-mode",
    "response-format",
    "json-schema",
    "constrained-decoding",
    "agents"
  ],
  "updated": "2026-07-02",
  "canonical": "https://changegamer.ai/resources/structured-outputs-and-json-mode",
  "markdown": "https://changegamer.ai/resources/structured-outputs-and-json-mode.md",
  "outline": [
    {
      "depth": 2,
      "text": "Key facts",
      "anchor": "key-facts"
    },
    {
      "depth": 2,
      "text": "Which mode to use",
      "anchor": "which-mode-to-use"
    },
    {
      "depth": 2,
      "text": "Provider mechanisms",
      "anchor": "provider-mechanisms"
    },
    {
      "depth": 3,
      "text": "OpenAI",
      "anchor": "openai"
    },
    {
      "depth": 3,
      "text": "Anthropic (Claude)",
      "anchor": "anthropic-claude"
    },
    {
      "depth": 3,
      "text": "Google Gemini",
      "anchor": "google-gemini"
    },
    {
      "depth": 3,
      "text": "Self-hosted / open models",
      "anchor": "self-hosted-open-models"
    },
    {
      "depth": 2,
      "text": "Provider comparison",
      "anchor": "provider-comparison"
    },
    {
      "depth": 2,
      "text": "Common failure modes",
      "anchor": "common-failure-modes"
    },
    {
      "depth": 2,
      "text": "SDK helpers",
      "anchor": "sdk-helpers"
    },
    {
      "depth": 2,
      "text": "Verified sources",
      "anchor": "verified-sources"
    }
  ],
  "related": [
    {
      "slug": "reliable-tool-calling",
      "title": "Reliable Tool Calling and Structured Outputs",
      "description": "How providers guarantee schema-valid tool calls and structured output — mechanisms, failure modes, and mitigations — for production agent builders.",
      "url": "https://changegamer.ai/resources/reliable-tool-calling"
    },
    {
      "slug": "agent-cost-latency-optimization",
      "title": "Agent Cost and Latency Optimization",
      "description": "Practitioner reference for reducing the cost and latency of production AI agents: the compounding model, token-level levers (caching, pruning), request-level levers (Batch API, parallelism), model-level levers (routing, reasoning-effort controls), and architecture-level levers (step reduction, semantic caching, code offloading).",
      "url": "https://changegamer.ai/resources/agent-cost-latency-optimization"
    },
    {
      "slug": "agent-identity-authentication",
      "title": "Agent Identity and Authentication",
      "description": "How autonomous agents prove who they are and get authorized to act: workload identity vs. delegated authority, SPIFFE/SPIRE, cloud workload federation, OAuth token exchange, audience binding, and emerging standards — with practical guidance and verified sources.",
      "url": "https://changegamer.ai/resources/agent-identity-authentication"
    },
    {
      "slug": "agent-memory-context",
      "title": "Agent Memory and Context Management",
      "description": "Architecture reference for agent memory: types (working, long-term, episodic, semantic, procedural), context-management techniques (summarization, RAG, sliding windows, prompt caching), storage substrates, and memory frameworks — with security notes and cross-links to related guides.",
      "url": "https://changegamer.ai/resources/agent-memory-context"
    }
  ],
  "body": "Structured outputs force an LLM to return valid JSON that conforms to a specific schema, without requiring tool calls. This is distinct from tool calling: the model's *final reply* is the schema-constrained object, not an intermediate function invocation. See /resources/reliable-tool-calling for the tool-calling primitive and how they interact.\n\n## Key facts\n\n- Structured outputs constrain the model's own final reply into a schema-valid object, which is a distinct mechanism from tool calling's intermediate function invocation.\n- True schema-constrained decoding (`json_schema` / strict mode) physically prevents schema-violating tokens, whereas legacy JSON mode only asks the model for syntactically valid JSON with no schema enforced — which is why OpenAI now treats plain JSON mode as legacy.\n- Each provider enforces and limits schemas differently: OpenAI's strict mode disallows default values and root-level `anyOf`, Anthropic's GA structured outputs support `anyOf`/`$ref` but not recursion, and Gemini splits into an OpenAPI-subset schema for older models versus a fuller JSON Schema for 2.5+.\n- Self-hosted serving stacks push schema enforcement down to token sampling via grammars — vLLM and SGLang default to the XGrammar library for this, while llama.cpp relies on its own GBNF grammar format.\n- Even under a strict schema guarantee, output can still come back non-compliant in practice if generation hits a token-limit truncation or a safety refusal — so the stop reason always needs checking.\n- First-request latency can rise because of grammar compilation, which is mitigated by a warm-up call and avoiding frequent schema churn.\n- Major SDKs from OpenAI, Anthropic, and Gemini all offer helpers that derive the schema directly from a Pydantic or Zod model and parse the result.\n\n## Which mode to use\n\n**Structured outputs / `json_schema`** — the provider's constrained decoder physically prevents the model from emitting tokens that violate your schema. Guarantees schema compliance given no refusal and no token-limit truncation.\n\n**JSON mode / `json_object`** — the model is merely instructed to produce syntactically valid JSON; no schema is enforced, so it may omit fields, use wrong types, or add keys. OpenAI now labels this mode legacy — prefer `json_schema` with `strict: true`.\n\n## Provider mechanisms\n\n### OpenAI\nSet `response_format: { type: \"json_schema\", json_schema: { name, schema, strict: true } }`. With `strict: true`, constrained decoding guarantees the output matches the schema. Requirements: every object must set `additionalProperties: false`; every property must appear in `required` (make optional fields a nullable type union). Supported on gpt-4o (2024-08-06+), GPT-4.1, GPT-5, and the o-series. Older snapshots support only the weaker `json_object` mode.\n\n**Schema subset — not supported by strict mode:** `default` values (the call fails); root-level `anyOf` (use a nullable type union instead); numeric/string constraints like `minimum`/`pattern` (accepted but not enforced by the decoder); very deep nesting / very large property counts. `response_format` and `tools` are mutually exclusive in a single call.\n\n### Anthropic (Claude)\nNative structured outputs shipped in late 2025 and are now GA on current models — no beta header required. Set `output_config: { format: { type: \"json_schema\", schema } }`; the schema is compiled to a grammar and generation is constrained at decode time. Also supports `strict: true` on tool definitions for guaranteed tool-input validation. Notes: `anyOf`/`$ref`/`$defs` supported; numeric range and recursive schemas are not; per-request limits apply (caps on tool count, optional/union parameters, and grammar-compilation time — check the docs for current values); a schema change invalidates the grammar cache, adding first-request latency; `stop_reason: \"refusal\"` or `\"max_tokens\"` bypasses the guarantee, so always check `stop_reason`.\n\n### Google Gemini\nSet `responseMimeType: \"application/json\"` together with a schema. Two schema formats: **`responseSchema`** (an OpenAPI-based subset; no `anyOf`/`$ref`; Gemini 1.5/2.0) and **`responseJsonSchema`** (full JSON Schema with `anyOf`/`$ref`; Gemini 2.5+, accepts Pydantic/Zod schemas). Known limitation: in some 2.5 snapshots structured output fails when tool-call messages are in history; overly complex schemas can trigger a 400 — shorten property names, reduce enum size, or flatten nested arrays.\n\n### Self-hosted / open models\nPass the schema at the serving layer, not in the prompt. vLLM and SGLang accept `guided_json` (JSON Schema), `guided_grammar` (EBNF/GBNF), or `guided_regex`. XGrammar (the default backend for vLLM/SGLang) compiles JSON Schema to a pushdown automaton with low per-token overhead; Outlines (dottxt-ai) uses FSM-based masking but does not support recursive schemas — use XGrammar or llguidance for those. llama.cpp uses GBNF grammars.\n\n## Provider comparison\n\n| Provider | Parameter | Strict guarantee? | anyOf | Defaults | Recursion |\n|---|---|---|---|---|---|\n| OpenAI (gpt-4o+, GPT-4.1+, GPT-5) | `response_format: json_schema, strict:true` | Yes | Root not allowed; use type unions | No (call fails) | No (depth cap) |\n| OpenAI (legacy `json_object`) | `response_format: json_object` | Syntax only | N/A | N/A | N/A |\n| Anthropic (recent Claude models) | `output_config.format: json_schema` | Yes | Yes | Yes | No |\n| Gemini 1.5 / 2.0 | `responseMimeType` + `responseSchema` | Subset | No | N/A | No |\n| Gemini 2.5+ | `responseMimeType` + `responseJsonSchema` | Full JSON Schema | Yes | Yes | Limited |\n| vLLM / SGLang (XGrammar) | `guided_json` | Yes | Yes | N/A | Yes |\n| llama.cpp | `grammar` (GBNF) | Yes | Via grammar | N/A | Yes |\n\n## Common failure modes\n\n- **API rejected (400)** — schema uses unsupported features (`default`, numeric ranges, recursive `$ref`). Strip unsupported keywords (most SDKs do this automatically).\n- **Output truncated mid-object** — `stop_reason: \"max_tokens\"`; the schema cannot prevent an early stop. Always check the stop reason; raise `max_tokens` or simplify the schema.\n- **Schema ignored despite no error** — you used `json_object` instead of `json_schema` strict.\n- **Model refuses** — a safety refusal overrides the schema (`stop_reason: \"refusal\"`); treat it as a first-class error branch.\n- **First request slow** — grammar compilation; warm up with a test call and avoid schema churn.\n\n## SDK helpers\n\nOpenAI `client.beta.chat.completions.parse(response_format=PydanticModel)` (Python) / `zodResponseFormat(schema, name)` (TS); Anthropic `messages.parse(output_format=PydanticModel)` (on `messages.create` itself the canonical parameter is `output_config.format` — the top-level `output_format` create-param is deprecated); Gemini (google-genai) accepts a Pydantic class directly as `response_schema`. These auto-derive the schema and parse the result.\n\n## Verified sources\n\n- Anthropic structured outputs docs: https://platform.claude.com/docs/en/build-with-claude/structured-outputs\n- Google Gemini structured output docs: https://ai.google.dev/gemini-api/docs/structured-output\n- OpenAI Structured Outputs guide: https://developers.openai.com/api/docs/guides/structured-outputs\n- XGrammar paper (constrained generation, arXiv:2411.15100): https://arxiv.org/abs/2411.15100\n- Outlines (dottxt-ai): https://github.com/dottxt-ai/outlines",
  "sources": [
    "https://platform.claude.com/docs/en/build-with-claude/structured-outputs",
    "https://ai.google.dev/gemini-api/docs/structured-output",
    "https://developers.openai.com/api/docs/guides/structured-outputs",
    "https://arxiv.org/abs/2411.15100",
    "https://github.com/dottxt-ai/outlines"
  ]
}