{ "slug": "agent-reliability-in-production", "title": "Agent Guardrails and the AI Agent Reliability Playbook", "description": "Agent guardrails plus the eleven other disciplines that make an AI agent reliable in production: tool calling, retries, durable execution and rollout.", "kind": "pillar", "order": 0, "target_query": "agent guardrails", "secondary_queries": [ "how to make an AI agent reliable", "durable execution", "tool calling llm", "structured outputs vs function calling", "AI agent reliability best practices", "production-ready AI agent checklist", "agent tool calling retries and idempotency", "how to evaluate AI agent reliability", "AI agent incident response runbook" ], "tags": [ "agents", "reliability", "production", "tool-calling", "observability", "evaluation", "durable-execution" ], "published": "2026-08-25", "updated": "2026-09-04", "words": 5776, "estimated_tokens": 7682, "premium": false, "rights": { "access": "free", "note": "Editorial guides are always free and never part of the licensed corpus.", "license": "https://changegamer.ai/license.xml", "pricing": "https://changegamer.ai/api/pricing.json", "payment": "https://changegamer.ai/api/payment.json" }, "license": "https://changegamer.ai/license.xml", "citation": "ChangeGamer (2026-08-25). Agent Guardrails and the AI Agent Reliability Playbook. ChangeGamer. https://changegamer.ai/articles/agent-reliability-in-production (updated 2026-09-04).", "bibtex": "@misc{changegamer_agent_reliability_in_production, title = {Agent Guardrails and the AI Agent Reliability Playbook}, publisher = {ChangeGamer}, year = {2026}, url = {https://changegamer.ai/articles/agent-reliability-in-production}, note = {Updated 2026-09-04}}", "canonical": "https://changegamer.ai/articles/agent-reliability-in-production", "markdown": "https://changegamer.ai/articles/agent-reliability-in-production.md", "takeaways": [ "AI agent reliability is not one property but a stack of twelve disciplines — tool-calling contracts, structured outputs, retries, idempotency, timeouts, durable execution, guardrails, CI evaluation, observability, incident response, staged rollout, and circuit breakers — and a demo agent that skips most of them will fail in ways a single-turn chatbot never does.", "Tool-call reliability starts with strict schema enforcement: OpenAI strict mode, Anthropic tool `strict: true` plus Structured Outputs, and Gemini `ANY`-mode function calls each physically constrain decoding so the model cannot emit a schema-violating call, as documented as of August 2026 in the reliable-tool-calling reference.", "Every side-effecting tool call an agent makes needs an idempotency key derived from the run ID and step index, not from wall-clock time or a fresh random value, so a retry after a crash or a rate-limit backoff lands as a no-op instead of a duplicate charge or duplicate email.", "A durable execution engine — Temporal, Restate, DBOS, Inngest, or a cloud-native equivalent — is the correct tool for any agent task that spans more than one process lifetime, because it persists a checkpoint log and resumes from the last completed step instead of restarting from zero after a crash.", "Production readiness for an agent means its evaluation, observability, guardrails, and rollback all work independently of which underlying model is running, with a canary release and automatic rollback on behavioral regression rather than only on crashes.", "Incident response for an agent needs a written runbook per failure class before go-live — runaway cost, prompt regression, tool outage, security incident — because an agent that fails without a pre-defined response burns budget and trust in the minutes before anyone notices." ], "outline": [ { "depth": 2, "text": "What are agent guardrails, and what does \"reliable\" mean for an AI agent?", "anchor": "what-are-agent-guardrails-and-what-does-reliable-mean-for-an-ai-agent", "url": "https://changegamer.ai/articles/agent-reliability-in-production#what-are-agent-guardrails-and-what-does-reliable-mean-for-an-ai-agent" }, { "depth": 2, "text": "Why do tool-calling contracts matter more than prompt engineering?", "anchor": "why-do-tool-calling-contracts-matter-more-than-prompt-engineering", "url": "https://changegamer.ai/articles/agent-reliability-in-production#why-do-tool-calling-contracts-matter-more-than-prompt-engineering" }, { "depth": 3, "text": "Known failure modes and fixes", "anchor": "known-failure-modes-and-fixes", "url": "https://changegamer.ai/articles/agent-reliability-in-production#known-failure-modes-and-fixes" }, { "depth": 2, "text": "When should you use structured outputs instead of tool calling?", "anchor": "when-should-you-use-structured-outputs-instead-of-tool-calling", "url": "https://changegamer.ai/articles/agent-reliability-in-production#when-should-you-use-structured-outputs-instead-of-tool-calling" }, { "depth": 2, "text": "How should an agent handle retries and rate limits?", "anchor": "how-should-an-agent-handle-retries-and-rate-limits", "url": "https://changegamer.ai/articles/agent-reliability-in-production#how-should-an-agent-handle-retries-and-rate-limits" }, { "depth": 2, "text": "Why does idempotency matter more for agents than for a simple API call?", "anchor": "why-does-idempotency-matter-more-for-agents-than-for-a-simple-api-call", "url": "https://changegamer.ai/articles/agent-reliability-in-production#why-does-idempotency-matter-more-for-agents-than-for-a-simple-api-call" }, { "depth": 2, "text": "What timeout should an agent use for a tool call or LLM step?", "anchor": "what-timeout-should-an-agent-use-for-a-tool-call-or-llm-step", "url": "https://changegamer.ai/articles/agent-reliability-in-production#what-timeout-should-an-agent-use-for-a-tool-call-or-llm-step" }, { "depth": 2, "text": "What is durable execution and when do you actually need it?", "anchor": "what-is-durable-execution-and-when-do-you-actually-need-it", "url": "https://changegamer.ai/articles/agent-reliability-in-production#what-is-durable-execution-and-when-do-you-actually-need-it" }, { "depth": 2, "text": "How do guardrails keep an agent's behavior reliable, not just safe?", "anchor": "how-do-guardrails-keep-an-agent-s-behavior-reliable-not-just-safe", "url": "https://changegamer.ai/articles/agent-reliability-in-production#how-do-guardrails-keep-an-agent-s-behavior-reliable-not-just-safe" }, { "depth": 2, "text": "How do you evaluate whether an agent is reliable enough to ship?", "anchor": "how-do-you-evaluate-whether-an-agent-is-reliable-enough-to-ship", "url": "https://changegamer.ai/articles/agent-reliability-in-production#how-do-you-evaluate-whether-an-agent-is-reliable-enough-to-ship" }, { "depth": 3, "text": "Testing agents in CI without flaky, expensive builds", "anchor": "testing-agents-in-ci-without-flaky-expensive-builds", "url": "https://changegamer.ai/articles/agent-reliability-in-production#testing-agents-in-ci-without-flaky-expensive-builds" }, { "depth": 2, "text": "What should an agent's observability system capture?", "anchor": "what-should-an-agent-s-observability-system-capture", "url": "https://changegamer.ai/articles/agent-reliability-in-production#what-should-an-agent-s-observability-system-capture" }, { "depth": 2, "text": "How do you roll out a new agent version without breaking production?", "anchor": "how-do-you-roll-out-a-new-agent-version-without-breaking-production", "url": "https://changegamer.ai/articles/agent-reliability-in-production#how-do-you-roll-out-a-new-agent-version-without-breaking-production" }, { "depth": 2, "text": "What happens when an agent fails in production, and how should the team respond?", "anchor": "what-happens-when-an-agent-fails-in-production-and-how-should-the-team-respond", "url": "https://changegamer.ai/articles/agent-reliability-in-production#what-happens-when-an-agent-fails-in-production-and-how-should-the-team-respond" }, { "depth": 2, "text": "Why do circuit breakers and degraded mode matter for agent reliability?", "anchor": "why-do-circuit-breakers-and-degraded-mode-matter-for-agent-reliability", "url": "https://changegamer.ai/articles/agent-reliability-in-production#why-do-circuit-breakers-and-degraded-mode-matter-for-agent-reliability" }, { "depth": 2, "text": "A production reliability checklist", "anchor": "a-production-reliability-checklist", "url": "https://changegamer.ai/articles/agent-reliability-in-production#a-production-reliability-checklist" }, { "depth": 2, "text": "The full agent-reliability cluster", "anchor": "the-full-agent-reliability-cluster", "url": "https://changegamer.ai/articles/agent-reliability-in-production#the-full-agent-reliability-cluster" } ], "faq": [ { "question": "What are agent guardrails?", "answer": "Agent guardrails are runtime checks — applied at three checkpoints: on input before the model sees a request, on the model's output before the caller sees it, and immediately before a tool call fires — that enforce policy regardless of what the underlying model happens to produce on a given call. They catch different reliability failures at each checkpoint: malformed or off-policy input, schema or groundedness violations in the output, and tool calls outside an allowlist or exceeding safe parameter ranges. The practical pattern layers cheap deterministic checks under slower, more thorough ones such as an LLM-as-judge, and defaults to blocking or escalating a call to a human when a check is uncertain rather than defaulting to allow." }, { "question": "What makes an AI agent unreliable in the first place?", "answer": "An AI agent becomes unreliable when any of its failure surfaces lacks a mechanical safeguard: malformed or hallucinated tool calls that were never schema-validated, retries that duplicate a side effect because no idempotency key exists, a crash mid-task that loses all progress because there is no durable checkpoint, or a behavioral regression that ships to every user at once because there was no canary. Each of these is a known, named failure mode with a standard fix, not an inherent property of language models." }, { "question": "How do I make tool calling reliable for an AI agent?", "answer": "Reliable tool calling combines provider-side schema enforcement (OpenAI strict mode, Anthropic strict tool definitions, Gemini ANY-mode function calls, or grammar-based constrained decoding for self-hosted models) with defensive validation on your side — reject unknown tool names, enforce required fields before executing, keep schemas shallow with few required fields, and add a validate-and-resubmit loop that sends validation errors back to the model as a correction request rather than crashing the agent." }, { "question": "What is the difference between retries and idempotency for AI agents?", "answer": "Retries are the mechanism that resends a failed request after a transient error such as a rate limit or network timeout, using exponential backoff with jitter capped by the provider's Retry-After header; idempotency is the guarantee that resending the same request — whether by your own retry logic or by a crash-recovery replay — never duplicates the underlying side effect, which requires a stable idempotency key derived from the run and step identity rather than from time or a fresh random value." }, { "question": "How do you evaluate whether an AI agent is reliable enough to ship?", "answer": "Evaluating agent reliability requires running an offline eval suite against your own task distribution — not just public benchmarks — scored on task success rate, tool-call accuracy, cost per task, and run-to-run consistency using multiple trials (pass^k, not a single pass/fail), then gating every promotion behind that suite in CI and rolling the change out behind a canary with automatic rollback on metric regression rather than only on crashes." }, { "question": "Do I need durable execution for every AI agent?", "answer": "No — durable execution is necessary only for agent tasks that can outlive a single process lifetime, such as workflows that wait on human approval, run for minutes to hours, or must survive a server restart without losing progress; a short-lived, single-turn agent call that completes in seconds within one request handler does not need a durable execution engine, though it still needs ordinary retry and timeout handling." } ], "body": "\n## What are agent guardrails, and what does \"reliable\" mean for an AI agent?\n\nAgent guardrails are runtime checks — on input, on the model's output, and immediately before a tool call fires — that enforce policy regardless of what the underlying model happens to produce on a given call. They are one of twelve disciplines that make an AI agent reliable: predictable and recoverable under the failure conditions that are certain to occur — malformed model output, a rate-limited API, a crashed process, a bad code deploy — not merely working on a demo run. Chat-completion reliability and agent reliability are different disciplines. A single-turn chatbot fails softly: a bad response is annoying but bounded. An agent fails compoundingly: a hallucinated tool call executes against a real system, a missed retry duplicates a database write, a crash mid-workflow loses hours of progress, and a bad deploy propagates to every user before anyone notices. Reliability engineering for agents is the set of mechanical safeguards that convert those compounding failures into bounded, recoverable, observable ones.\n\nThis guide treats reliability as a stack of twelve disciplines, roughly in the order a request travels through an agent and then through its release pipeline:\n\n1. **Tool-calling contracts** — does the model reliably call the right tool with valid arguments?\n2. **Structured outputs** — when you need a schema-valid final answer instead of a tool call, does the provider guarantee it?\n3. **Retries and rate limits** — how does the agent behave when a provider or tool returns a transient error?\n4. **Idempotency** — do retries and crash-recovery replays avoid duplicating side effects?\n5. **Timeouts** — how long does a step wait before the system gives up and reacts?\n6. **Durable execution** — does a long-running task survive a crash or restart?\n7. **Guardrails** — does the agent's behavior stay within policy regardless of what the model outputs?\n8. **Evaluation in CI** — is reliability measured and gated before every change ships?\n9. **Observability** — can you reconstruct what an agent did and why, after the fact?\n10. **Rollout** — does a new version reach production gradually, with automatic rollback?\n11. **Incident response** — does the team have a pre-written response for each failure class?\n12. **Circuit breakers and degraded mode** — does the system fail fast and gracefully instead of stalling or cascading?\n\nNone of these is exotic engineering; each has a standard pattern documented in the reference corpus this guide draws from. What changes production outcomes is doing all twelve deliberately instead of discovering the gaps one incident at a time.\n\n## Why do tool-calling contracts matter more than prompt engineering?\n\nTool-calling contracts matter more than prompt engineering because a single malformed tool call turns an otherwise-working agent step into an error-handling problem downstream, and no amount of prompt polish eliminates that risk the way schema enforcement does. As of August 2026, every major provider offers some form of constrained decoding that physically restricts which tokens the model can emit for a tool call or a structured final answer, rather than merely asking nicely: OpenAI's strict mode requires every object to set `additionalProperties: false` and every property to appear in `required`; Anthropic supports `strict: true` on individual tool definitions plus a separate Structured Outputs mechanism (`output_config`) for the final reply; Google Gemini forces compliance through `ANY`-mode function calls or a `responseSchema`/`responseJsonSchema` config; and self-hosted models push the same guarantee down to token sampling via grammars such as GBNF (llama.cpp), XGrammar (the default backend for vLLM, SGLang, TensorRT-LLM, and MLC-LLM), or Outlines.\n\n### Known failure modes and fixes\n\nEven with constrained decoding enabled, real failures still recur in predictable shapes: a hallucinated tool name outside the declared set, a missing required argument, extraneous fields, unparsable JSON when strict mode isn't enabled, wrong field types, over-calling or under-calling tools, and out-of-order parallel calls with unstated dependencies. Each pairs with a known fix — validate the returned tool name against your schema before executing, enforce `required`/`additionalProperties: false`, strip unknown keys defensively, wrap parsing in try/catch with a retry-and-repair loop that sends the validation error back to the model as a correction request, prefer enums and `const` values over free-text fields, and set `disable_parallel_tool_use: true` (Anthropic) or force sequential `tool_choice` when call order matters. The Berkeley Function Calling Leaderboard (BFCL), maintained by the Gorilla team at UC Berkeley, is the standard benchmark for this failure class, scoring both single and simultaneous tool calls via AST comparison; its fourth iteration (current as of August 2026) folds in multi-step agentic tasks rather than testing isolated calls only. The full failure-mode table and mitigation reference lives in [reliable tool calling and structured outputs](/resources/reliable-tool-calling).\n\nThe cross-cutting mitigation that matters most in practice: keep schemas shallow with a minimal set of required fields. Every optional field is a reliability risk, because it is one more place the model can omit, mistype, or hallucinate a value — a lesson that shows up just as often in MCP tool handlers as in first-party function calling (see [common MCP server failure modes](/articles/mcp-server-failure-modes) for the crash-recovery and malformed-call-handling side of this same problem at the protocol layer).\n\n## When should you use structured outputs instead of tool calling?\n\nUse structured outputs when you need the model's own final answer in machine-parseable form and use tool calling when you need the model to invoke an external function — the two mechanisms are orthogonal and can be required together or separately. Tool calling produces an intermediate instruction for your application to execute (a named function plus arguments); structured outputs constrain the model's concluding response itself into a schema-valid object, with no function invoked. A classification task that just needs `{\"category\": \"billing\", \"confidence\": 0.92}` back from the model wants structured outputs, not a tool call; an agent that needs to actually query a database or send an email wants a tool call.\n\nThe provider mechanisms differ in what schema subset they support, which matters when you design a contract meant to survive a provider swap:\n\n| Provider | Mechanism | Strict guarantee | `anyOf` | Recursion |\n|---|---|---|---|---|\n| OpenAI (gpt-4o 2024-08-06+, GPT-4.1, GPT-5, o-series) | `response_format: json_schema, strict: true` | Yes | Root-level not allowed; use nullable type unions | No (depth cap) |\n| Anthropic (current models, GA as of August 2026) | `output_config.format: json_schema` | Yes | Yes | No |\n| Gemini 1.5 / 2.0 | `responseSchema` (OpenAPI subset) | Subset only | No | No |\n| Gemini 2.5+ | `responseJsonSchema` (full JSON Schema) | Yes | Yes | Limited |\n| vLLM / SGLang (XGrammar) | `guided_json` | Yes | Yes | Yes |\n| llama.cpp | `grammar` (GBNF) | Yes | Via grammar | Yes |\n\nTwo failure modes catch teams even under a strict-mode guarantee. First, **truncation and refusal bypass the schema**: a response cut off by `stop_reason: \"max_tokens\"` or an explicit safety refusal is not schema-compliant even though strict mode was enabled, so always check the stop reason as a first-class error branch rather than assuming a 200 response means a valid object. Second, **first-request latency rises from grammar compilation** the first time a new schema is used — mitigate with a warm-up call and by avoiding frequent schema churn. OpenAI now treats plain JSON mode (`json_object`) as legacy precisely because it only asks for syntactically valid JSON with no schema enforced; prefer `json_schema` with `strict: true` wherever the provider supports it. The full per-provider schema-subset comparison is in [structured outputs and JSON mode](/resources/structured-outputs-and-json-mode).\n\n## How should an agent handle retries and rate limits?\n\nAn agent should treat a 429 (Too Many Requests) as a signal to honor the provider's `Retry-After` header and wait that long before retrying — never retry immediately, and never treat rate limits as a bug to route around rather than a budget to respect. Provider limits stack across at least three windows: requests per minute (RPM), tokens per minute (TPM, sometimes split into separate input/output limits as Anthropic does), and a daily ceiling (TPD/RPD) that overrides any remaining per-minute headroom. Both OpenAI and Anthropic loosen these ceilings as account spend and age grow through numbered usage tiers, but exact thresholds shift over time — check your provider dashboard rather than hard-coding a number, and request a tier increase ahead of need rather than reacting to production 429s.\n\nThe standard backoff pattern is exponential backoff with full jitter — randomizing the wait uniformly between zero and an exponentially growing cap, while always enforcing the provider's mandated `Retry-After` as a hard floor:\n\n```\nMAX_RETRIES = 7\nBASE_DELAY = 1 # seconds\nCAP_DELAY = 60 # seconds\n\ndef call_with_backoff(request):\n for attempt in range(MAX_RETRIES):\n response = send(request)\n if response.status != 429:\n return response\n floor = int(response.headers.get(\"Retry-After\", 0))\n ceiling = min(CAP_DELAY, BASE_DELAY * 2 ** attempt)\n wait = max(floor, random_uniform(0, ceiling))\n sleep(wait)\n raise RateLimitExceeded(\"max retries reached\")\n```\n\nFull jitter matters at fleet scale: without it, many retrying clients synchronize their backoff schedules and hammer the API in the same instant, turning a transient rate limit into a retry storm. Before ever reaching a 429, well-behaved agents self-throttle proactively — track a local token budget from the `*-remaining` response headers, cap concurrent in-flight requests with a semaphore, and pre-estimate token cost to hold back requests that would exceed remaining headroom. Not every error deserves a retry: distinguish transient failures (network timeout, 429, 5xx) that should retry with backoff from terminal failures (400 bad request, business-logic error) that should escalate immediately — retrying a malformed request forever just delays the failure. For high-volume, non-interactive workloads — evals, bulk summarization, embedding a large corpus — both major providers offer a batch API on a separate rate-limit pool at roughly half the per-token price with same-day turnaround; it is the wrong tool for a latency-sensitive interactive agent but the right one for the CI evaluation suite discussed later in this guide. See [handling rate limits and retries](/resources/handling-rate-limits-and-retries) for the full header reference and tier-advancement details.\n\n## Why does idempotency matter more for agents than for a simple API call?\n\nIdempotency matters more for agents than for a stateless API call because an agent retries far more often — after rate limits, after transient tool errors, and after crash recovery — and every one of those retries risks re-executing a side effect that already happened. A crash can leave a step mid-flight, and a naive retry (or a durable-execution replay, discussed below) may attempt that step a second time. If the step is a database write, a payment charge, or an outbound email, a duplicate attempt is a duplicate side effect — a double charge, a double email, a corrupted counter.\n\nThe standard fix is a stable idempotency key derived from the workflow run ID and step index — never from wall-clock time or a freshly generated random value, both of which change on every retry and therefore cannot deduplicate anything. Pass that key to the downstream service (most payment, email, and messaging APIs accept an idempotency-key header) so a duplicate call becomes a no-op instead of a duplicate effect. This single pattern is the difference between \"the agent retried and it worked\" and \"the agent retried and now the customer has two invoices,\" and it applies whether the retry originates from your own backoff loop, from a durable execution engine's crash recovery, or from an MCP client resending a tool call after a timeout.\n\n## What timeout should an agent use for a tool call or LLM step?\n\nEvery step in an agent's execution — an LLM call, a tool call, a sub-agent hop — needs an explicit timeout tied to what a reasonable worst case looks like for that specific operation, not a single global value copied across every step; no universal number exists because the right timeout depends entirely on what the step does. A timeout that is too short cancels legitimate slow operations (a large document extraction, a complex database query) and converts them into retries that compound latency; a timeout that is too long lets one stuck dependency stall an entire agent run and burn budget while nobody notices.\n\nTwo design choices compound with timeouts and are worth setting deliberately rather than defaulting:\n\n- **Bound the retry budget in wall-clock time, not just attempt count.** A capped exponential backoff with several retries can still take a minute or more to exhaust; if the caller (a user-facing request, an upstream orchestrator) has its own SLA, the agent's total retry-plus-timeout budget must fit inside it, or the caller times out first and the retries become wasted work.\n- **Treat a timeout as a distinct outcome from an error.** A step that times out has an unknown final state — the underlying operation may have completed on the far side even though the response never arrived. This is exactly the scenario idempotency keys exist for: a timed-out payment call retried with the same key is safe to resend even though you cannot be sure whether the first attempt succeeded.\n\nDurable execution engines (next section) make per-step timeout policy — maximum attempts, initial interval, backoff multiplier, and a non-retryable error list — a first-class configuration rather than something reimplemented ad hoc in each workflow, as detailed in [durable execution for long-running agents](/resources/durable-execution-for-agents).\n\n## What is durable execution and when do you actually need it?\n\nDurable execution is the programming model that lets a long-running agent workflow survive a crash, a restart, or a wait for human approval by persisting every meaningful step to a log before moving on, so that on resume the engine reconstructs state from stored records instead of re-executing real-world actions a second time. You need it for any agent task that can outlive a single process lifetime — a workflow that pauses for human approval, that runs for minutes to hours across multiple tool calls, or that must survive a server restart without losing progress. You do not need it for a short-lived, single-turn agent call that completes within one request handler in seconds; that case still needs ordinary retry and timeout handling, just not a checkpoint log.\n\nTwo constraints make durable execution work correctly, and violating either one is the most common way teams misuse it:\n\n- **Workflow code must be deterministic.** Because resuming re-runs the orchestration function, that function must produce the same sequence of commands given the same recorded inputs. Non-deterministic operations — clock reads, randomness, direct LLM or API calls — cannot live inside the workflow logic itself; they belong in a separately recorded unit (called an activity, step, or handler depending on the engine) that executes outside the replay path and is injected from its stored result on resume.\n- **Side-effecting calls need the idempotency key described above**, because a crash can cause a step that was mid-flight to be attempted again during replay.\n\nCurrent engines fall into four architectural shapes, and the right choice usually follows existing platform commitment rather than an abstract feature comparison:\n\n| Situation | Recommended approach |\n|---|---|\n| Already on AWS | AWS Step Functions (Standard Workflows) — managed state machine, native integration |\n| Already on Azure | Azure Durable Functions — event-sourcing model, first-party |\n| Already on GCP | GCP Workflows — managed state machine, callback-based human-in-the-loop |\n| Already on Cloudflare Workers | Cloudflare Workflows — Durable-Object-backed, co-located with edge compute |\n| Need portable, code-first durability; willing to self-host | Temporal (mature, large ecosystem), Restate (lighter, suspension-native), or DBOS (Postgres-only dependency) |\n| Managed serverless, TypeScript/JavaScript-first | Inngest — step memoization, official self-hosting path also now exists |\n| Already using LangGraph | LangGraph checkpointers + `interrupt()` — lighter-weight, single-graph fault tolerance, not full cross-service orchestration |\n| Already using OpenAI Agents SDK, need memory continuity but not crash-resume | Agents SDK Sessions — simplest path; add a dedicated engine if mid-run durability is required |\n\nA durably parked workflow — one suspended on an external signal such as a human approval or a webhook — consumes no compute while waiting and restores full state the instant the signal arrives. This is architecturally distinct from a polling loop or a sleeping thread, and it is the mechanism that makes human-in-the-loop approval gates (see guardrails, below) actually durable rather than a fragile in-memory wait with a timeout attached.\n\n## How do guardrails keep an agent's behavior reliable, not just safe?\n\nGuardrails keep an agent's behavior reliable by enforcing policy at runtime regardless of what the underlying model happens to output on a given call, which matters for reliability specifically because model behavior is inherently variable across calls even at fixed settings. In reliability terms — as distinct from the security-hardening question of defending against an adversarial attacker, which belongs to a different discipline — the guardrail question is simpler: does the agent behave correctly and predictably, and does a tool call that would produce an incorrect or unapproved action get stopped before it executes rather than after?\n\nGuardrails apply at three checkpoints, each catching a different reliability failure:\n\n- **Input guardrails**, before the model sees a request — catch malformed or off-policy input before it can produce a malformed downstream action.\n- **Output guardrails**, after the model replies but before the caller sees it — catch schema or format violations and groundedness failures (does the answer stay within the intended scope?).\n- **Action guardrails**, immediately before a tool call fires — catch calls outside an allowlist, calls exceeding defined parameter ranges, and irreversible or high-stakes operations that should require human approval rather than executing autonomously.\n\nNo single check catches everything a model might produce, so the practical pattern layers cheap, fast checks (regex and deterministic validators for format and known patterns) underneath slower, more thorough ones (a classifier for known categories, an LLM-as-judge for nuance), and defaults to blocking or escalating when a check is uncertain rather than defaulting to allow. Every guardrail verdict — allowed or blocked, which rule fired, what score it produced — should be logged against the run's trace ID, both to debug false positives that degrade usability and false negatives that provide false confidence, and to feed the observability discipline covered next. A single tool call should touch at most two of three properties — processing untrusted input, accessing a sensitive system, or changing external state — never all three at once; a call that would touch all three (say, acting on instructions found in a scraped web page to modify production data) is exactly the shape of action that belongs behind a human-approval gate rather than autonomous execution. The full checkpoint taxonomy and tooling landscape is in [guardrails and safety filters for agents](/resources/agent-guardrails).\n\n## How do you evaluate whether an agent is reliable enough to ship?\n\nEvaluating agent reliability requires scoring multi-step trajectories against your own task distribution, not a single-turn response or a public benchmark alone, because a model that scores well on a chat benchmark can still fail badly as an agent if it cannot recover from tool errors, maintain state across steps, or complete long-horizon tasks consistently. Three properties distinguish agent evaluation from ordinary LLM evaluation:\n\n- **Multi-step trajectories, not single-turn correctness.** Errors compound across steps, so partial-credit metrics — did the agent complete 8 of 10 sub-steps correctly? — carry more signal than a binary pass/fail on the final outcome alone.\n- **Reliability across repeated runs, not a single trial.** The same prompt can produce different tool-call sequences across runs, so a single pass/fail measurement is misleading. Use `pass@k` (at least one of k trials succeeds — a ceiling-capability metric) and `pass^k` (all k trials succeed — a consistency metric, used as tau-bench's primary metric) rather than reporting one run as the answer.\n- **Cost and latency as first-class metrics alongside accuracy.** An agent that succeeds but costs substantially more or takes substantially longer than a simpler baseline is not actually ready to ship; token spend and wall-clock time per task belong in the same eval report as success rate.\n\nPublic benchmarks — SWE-bench for software-engineering tasks, GAIA for cross-tool generalist reasoning, BFCL for tool-call correctness, WebArena for web navigation, and others — are useful proxies but reliably diverge from production performance, so build an eval suite against your own task distribution and guard against benchmark contamination rather than shipping on leaderboard position alone. For tool-call correctness specifically, exact-match or AST-based comparison against a reference answer is more reliable than LLM-as-judge, which carries known biases (position bias, verbosity bias, self-preference when a model judges its own family's output) — reserve LLM-as-judge for open-ended subjective quality and use ground-truth matching for anything with a checkable structure. See [evaluating AI agents](/resources/evaluating-ai-agents) for the full benchmark reference table.\n\n### Testing agents in CI without flaky, expensive builds\n\nTesting an agent in CI resolves the tension between non-deterministic models and deterministic pipelines by testing different layers at different levels rather than trying to make every test reproducible end to end. A three-layer pyramid works in practice: **Layer 1** is ordinary deterministic unit tests of the code around the model — tool functions, parsers, schema validators, retry logic — with the LLM client mocked entirely, run on every commit. **Layer 2** uses cassette/VCR-style recorded fixtures: the first run hits the real API and serializes the exchange to a committed file; every subsequent run replays it, fast and network-independent, re-recorded only when prompts or schemas change. **Layer 3** is a small, hand-curated set of live smoke tests against the real model, gated on a separate nightly or pre-release CI job rather than per-commit, because they are inherently slower and flakier and should never block a PR.\n\nTwo techniques worth calling out because they are commonly misused: setting `temperature=0` reduces variance but does not guarantee bit-for-bit identical outputs across runs (floating-point non-associativity from GPU batching and MoE routing still produces different tokens in different batch contexts), so never rely on it as a substitute for mocking or cassette replay; and snapshot testing of tool-call sequences — asserting on the structure and argument values of the expected tool-call trajectory, not on free-text reasoning — surfaces unintended trajectory changes in CI before they reach production. For probabilistic Layer 3 tests, use pass@k thresholds (e.g., pass@5 with at least 4 successes) rather than a single run, and quarantine flaky tests into the nightly suite rather than masking them with automatic retries. See [testing AI agents in CI](/resources/testing-ai-agents) for the tooling reference (VCR.py, pytest-recording, promptfoo, DeepEval).\n\n## What should an agent's observability system capture?\n\nAn agent's observability system should capture a full trace tree — every LLM call, tool call, retrieval step, and sub-agent invocation as a nested span under one stable trace ID — because flat, timestamped log lines cannot answer \"why did the agent do that?\" for a non-deterministic, branching run that may fan out into dozens of calls. The OpenTelemetry GenAI semantic conventions (formed by the GenAI SIG in April 2024, still labeled Development status as of July 2026 but already adopted by leading observability vendors) give this a vendor-neutral vocabulary: a **trace** is one complete agent run identified by a stable `trace_id` propagated across all child operations including sub-agents, and a **span** is one discrete operation — an LLM call, a tool call, a sub-agent hop — nested to form the full tree.\n\nAt minimum, capture per span: token usage converted into cost, latency for the step and for the overall trace, tool-call inputs and outputs (with PII redacted before logging), and the original exception attached to whichever span failed. Tooling splits between framework-agnostic OTel platforms — Langfuse, Arize Phoenix, OpenLLMetry, Logfire — and tracing built directly into a framework such as LangChain's LangSmith or the OpenAI Agents SDK's built-in trace processor. Stored spans double as raw material for two different feedback loops: sampling a slice of production traces builds evaluation datasets for judge- or metric-based scoring (closing the loop with the evaluation discipline above), and watching the same stream in real time flags anomalous behavior — a spike in tool errors, a cost outlier, a latency regression — as it happens rather than after a user complains. See [agent observability and tracing](/resources/agent-observability) for the OTel GenAI attribute reference and tooling landscape.\n\n## How do you roll out a new agent version without breaking production?\n\nRoll out a new agent version by gating the promotion behind your own task-distribution eval suite first, then releasing to a small canary slice of traffic with automatic rollback triggered by metric regression — not only by crashes. **Shadow mode**, where the new version runs and logs its outputs without serving them to real users, removes rollout risk entirely during initial validation, because you can compare the new version's behavior against the current one on live traffic before any user is exposed to a difference. A canary of roughly 5–10% of traffic then surfaces regressions that shadow mode's non-live conditions might miss, with automated rollback triggers watching for behavioral regression — a lower task success rate, a changed output distribution, a cost spike — rather than only outright errors.\n\nRollback itself needs to be a fast, tested operation, not an improvised one during an incident:\n\n- **Version code, prompt template, model pin, and tool contracts together** under one composite tag, so a rollback restores a known-good combination unambiguously rather than reverting one component while leaving others mismatched.\n- **Keep the previous version runnable for at least one canary period** so rollback is a one-command, minutes-not-rebuild operation.\n- **Test rollback in drills, not only during real incidents** — a rollback procedure nobody has exercised outside a crisis is a rollback procedure that fails when it matters most.\n\n## What happens when an agent fails in production, and how should the team respond?\n\nWhen an agent fails in production, the team's response should already be written down as a runbook per failure class before go-live, because deciding how to react while an incident is active costs both time and judgment that a pre-written plan preserves. Four failure classes cover most real incidents and each has a standard first response: **runaway cost** (hard per-session budget plus a kill switch that halts the agent rather than letting spend continue while someone investigates), **prompt or behavioral regression** (rollback to the previous known-good version plus traffic redirect, using the rollback procedure above), **tool or dependency outage** (a circuit breaker plus a degraded-mode fallback, covered next), and **security incident** (revoke credentials, disable the agent, and preserve traces for investigation independent of the production system that might otherwise be part of the compromise).\n\nTwo disciplines make every incident in every class resolvable rather than a mystery: **preserve traces on incident** — the full trace tree from observability, kept accessible for the investigation window even if the production system itself is degraded or rolled back — and **run a blameless post-mortem after every production incident**, treating the incident as a signal that a guardrail, an eval case, or a monitoring alert was missing rather than as an individual's mistake. The output of a good post-mortem is concrete: a new eval case added to the CI suite that would have caught the regression, a new guardrail rule, or a new alert threshold — closing the loop back into the evaluation and observability disciplines above so the same failure class cannot recur silently. The full ten-dimension ship gate, including rollout and rollback mechanics, is in [shipping AI agents to production](/resources/shipping-agents-to-production).\n\n## Why do circuit breakers and degraded mode matter for agent reliability?\n\nCircuit breakers and degraded mode matter because a permanently degraded dependency — a downed tool API, an exhausted daily rate limit, a database outage — should make an agent fail fast and predictably rather than retry forever and stall every request that touches it. A circuit breaker tracks the failure rate of a dependency and, once it crosses a threshold, stops sending requests to it for a cooldown period, returning an immediate failure (or falling back to a degraded response) instead of letting every caller independently retry and time out against a service that is not coming back soon. This is the natural complement to the retry-with-backoff pattern described earlier: backoff handles a single transient blip, while a circuit breaker handles a dependency that is down long enough that retrying is actively harmful — it burns budget, delays the caller, and can itself contribute to keeping the downstream service overloaded.\n\n**Degraded mode** is the fallback behavior a circuit breaker triggers: an agent that cannot reach its primary retrieval index falls back to a cached or smaller-scope answer with an explicit disclosure rather than fabricating one; an agent that cannot reach a preferred model routes to a secondary provider through a gateway rather than failing the whole request; an agent whose reranking stage is down serves unreranked first-stage results rather than blocking. Designing degraded mode deliberately — deciding in advance what \"good enough\" looks like when a dependency is unavailable — is what separates a system that gracefully loses some quality under partial outage from one that cascades a single dependency's failure into a total outage for every user.\n\n## A production reliability checklist\n\nBefore calling an agent production-ready, it should survive this list, organized by the twelve disciplines above:\n\n- Tool schemas use strict/constrained decoding where the provider supports it; unknown tool names and malformed arguments are rejected before execution\n- Structured-output calls check the stop reason, not just the HTTP status, because truncation and refusal both bypass the schema guarantee\n- All model and tool calls implement exponential backoff with full jitter, honoring `Retry-After` as a hard floor\n- Every side-effecting tool call carries an idempotency key derived from run ID and step index\n- Every step has an explicit timeout sized to its own worst case, with a wall-clock budget for the full retry sequence\n- Tasks spanning more than one process lifetime run on a durable execution engine with deterministic workflow code\n- Input, output, and action guardrails are layered independently of the model, with fail-closed behavior on uncertain verdicts\n- An offline eval suite scored against your own task distribution gates every promotion in CI, with reliability measured across multiple trials\n- Deterministic Layer 1/2 tests run on every commit; probabilistic Layer 3 smoke tests run nightly, never blocking a PR\n- Every run emits a full trace tree under one trace ID with per-span cost, latency, and redacted payloads\n- New versions ship behind a canary with automatic rollback on behavioral regression, and rollback is drilled, not just documented\n- A written runbook exists per failure class (cost, regression, outage, security) with traces preserved and a blameless post-mortem after every incident\n- Circuit breakers protect against permanently degraded dependencies, with a deliberately designed degraded-mode fallback\n\nNone of these disciplines is exotic in isolation — retries, idempotency keys, canaries, and CI gates are established engineering practice well outside AI. What is specific to agents is that a single request can touch all twelve at once: a tool call that needs a valid schema, a retry budget, an idempotency key, a timeout, a durable checkpoint if it runs long, a guardrail before it fires, an eval case that would catch a regression in it, a trace that explains it after the fact, a canary that limits its blast radius, a runbook if it fails, and a circuit breaker if its dependency goes down. Teams that treat reliability as this full stack, rather than as \"retry a few times and hope,\" are the ones whose agents survive contact with real traffic.\n\n## The full agent-reliability cluster\n\nTwelve sub-articles make up the agent-reliability cluster, each going deeper on one discipline than a single pillar reasonably can, and the cluster is complete as of August 2026:\n\n- [How to make AI agent tool calling reliable](/articles/tool-calling-contracts-for-ai-agents) turns this pillar's tool-calling section into per-model schema-enforcement mechanics with runnable examples, a full catalog of the ways a validated call still goes wrong, and where BFCL's AST-based scoring actually draws the line on a correct call.\n- [Structured outputs vs tool calling](/articles/structured-outputs-vs-tool-calling) turns the summary schema-subset table above into a framework for picking one mechanism over the other, with runnable schema code for every provider and a warm-up-call fix for the compilation delay a brand-new grammar pays on its very first call.\n- [How to make AI agent retries idempotent](/articles/retries-and-idempotency-for-ai-agents) goes past the `call_with_backoff` example above into which errors are even worth retrying and deriving a dedup key that survives a crash mid-call, not just a rate limit.\n- [When do AI agents need durable execution](/articles/durable-execution-for-ai-agents) expands the four-row situation table above into all ten vendors and frameworks it maps to, and the constraint that keeps a replayed workflow from re-running a real-world action a second time.\n- [How to design guardrails for AI agent reliability](/articles/agent-guardrails-for-reliability) expands the three-checkpoint list above into a layered pipeline of cheap deterministic checks under slower judgment calls, and the specific untrusted-input/sensitive-system/state-change combination that should pull a human into the loop before a tool call fires.\n- [How to evaluate AI agents in CI](/articles/evaluating-ai-agents-in-ci) expands the benchmark and CI-pyramid discussion above into the public-benchmark landscape as a diverging proxy and the `--record-mode=none` cassette-replay mechanics that keep per-commit CI honest without hitting a live model.\n- [How to roll out a new AI agent version safely](/articles/agent-rollout-and-rollback) goes past the canary-and-composite-tag summary above into where prompts should actually live, four ways to number a version, and how much traffic and patience an honest A/B test actually needs.\n- [How to build an incident response runbook for AI agent failures](/articles/agent-incident-response-runbooks) expands the four-failure-class table above into a three-signal triage step and the habit of exporting a trace before either recovery action gets a chance to overwrite it.\n- [How to set timeouts for AI agent tool calls](/articles/timeouts-and-deadlines-for-ai-agents) expands the pillar's single paragraph into three angles it has no room for: sizing each step type on its own worst case, reserving a shrinking slice of one overall budget as a chain runs, and passing whatever time remains down into a nested call.\n- [How to design a circuit breaker for AI agents](/articles/circuit-breakers-and-degraded-mode-for-ai-agents) expands the one-paragraph mention above into the breaker's three-state machine — letting traffic through, cutting it off, then testing recovery with a handful of probes — and a fallback response designed in advance for each dependency type rather than improvised during an outage.\n- [What should an AI agent's observability system capture](/articles/agent-observability-for-reliability) expands the one-paragraph trace/span mention above into the `gen_ai.*` wire vocabulary OpenTelemetry assigns to each field, the concrete list of what belongs on a span besides its timing, and a redaction rule for prompt and completion text specifically.\n- [The AI agent production reliability checklist](/articles/agent-reliability-production-checklist) turns the thirteen-bullet checklist above into a go/no-go gate per discipline, naming the specific evidence each one needs and the sibling article that owns its mechanics.\n", "cluster": { "id": "agent-reliability", "title": "Agent reliability in production", "description": "How to make an AI agent reliable — tool-calling contracts, structured outputs, retries and idempotency, timeouts, durable execution, guardrails, evaluation in CI, observability, incident response, and rollout.", "status": "complete", "pillar": { "slug": "agent-reliability-in-production", "title": "Agent Guardrails and the AI Agent Reliability Playbook", "description": "Agent guardrails plus the eleven other disciplines that make an AI agent reliable in production: tool calling, retries, durable execution and rollout.", "kind": "pillar", "order": 0, "html": "https://changegamer.ai/articles/agent-reliability-in-production", "markdown": "https://changegamer.ai/articles/agent-reliability-in-production.md", "json": "https://changegamer.ai/api/articles/agent-reliability-in-production.json" }, "articles": [ { "slug": "tool-calling-contracts-for-ai-agents", "title": "How to Make AI Agent Tool Calling Reliable", "description": "An operator playbook for tool-calling contracts: per-provider strict-mode config (OpenAI, Anthropic, Gemini, self-hosted grammars), the full failure-mode-to-fix table, what BFCL actually measures, and how to stop parallel tool calls from breaking a dependency chain.", "kind": "sub", "order": 1, "html": "https://changegamer.ai/articles/tool-calling-contracts-for-ai-agents", "markdown": "https://changegamer.ai/articles/tool-calling-contracts-for-ai-agents.md", "json": "https://changegamer.ai/api/articles/tool-calling-contracts-for-ai-agents.json" }, { "slug": "structured-outputs-vs-tool-calling", "title": "Structured Outputs vs Tool Calling: When to Use Each", "description": "A decision framework for structured outputs versus tool calling in AI agents, with runnable JSON Schema examples for OpenAI, Anthropic, Gemini, vLLM/SGLang, and llama.cpp, plus mitigation code for truncation, refusal, and grammar-compilation latency.", "kind": "sub", "order": 2, "html": "https://changegamer.ai/articles/structured-outputs-vs-tool-calling", "markdown": "https://changegamer.ai/articles/structured-outputs-vs-tool-calling.md", "json": "https://changegamer.ai/api/articles/structured-outputs-vs-tool-calling.json" }, { "slug": "retries-and-idempotency-for-ai-agents", "title": "How to Make AI Agent Retries Idempotent", "description": "A deep-dive on retrying agent tool calls safely: the transient-vs-terminal decision, why an agent side effect can fire before a failure signal reaches the caller, idempotency-key mechanics (run ID + step index), the unknown-outcome edge case, and where idempotency keys do not reach.", "kind": "sub", "order": 3, "html": "https://changegamer.ai/articles/retries-and-idempotency-for-ai-agents", "markdown": "https://changegamer.ai/articles/retries-and-idempotency-for-ai-agents.md", "json": "https://changegamer.ai/api/articles/retries-and-idempotency-for-ai-agents.json" }, { "slug": "durable-execution-for-ai-agents", "title": "When Do AI Agents Need Durable Execution?", "description": "A deep-dive on durable execution for AI agents: the persisted event log, the replay-determinism constraint, the four architectural shapes mapped across ten engines and frameworks, and a decision framework for when a durable execution engine is worth adding at all.", "kind": "sub", "order": 4, "html": "https://changegamer.ai/articles/durable-execution-for-ai-agents", "markdown": "https://changegamer.ai/articles/durable-execution-for-ai-agents.md", "json": "https://changegamer.ai/api/articles/durable-execution-for-ai-agents.json" }, { "slug": "agent-guardrails-for-reliability", "title": "How to Design Guardrails for AI Agent Reliability", "description": "An operator playbook for reliability guardrails: the three checkpoints (input, output, action), layering cheap checks under slow ones with a fail-closed default, the two-of-three-properties rule for when a tool call needs human approval, and logging every verdict against the run trace ID.", "kind": "sub", "order": 5, "html": "https://changegamer.ai/articles/agent-guardrails-for-reliability", "markdown": "https://changegamer.ai/articles/agent-guardrails-for-reliability.md", "json": "https://changegamer.ai/api/articles/agent-guardrails-for-reliability.json" }, { "slug": "evaluating-ai-agents-in-ci", "title": "How to Evaluate AI Agents in CI", "description": "An operator playbook for gating an AI agent release in CI: why agent eval needs trajectory-level scoring across the tasks it actually runs, how public benchmarks diverge as proxies, ground-truth vs LLM-as-judge tool-call scoring, and the three-layer test pyramid that keeps CI fast and non-flaky.", "kind": "sub", "order": 6, "html": "https://changegamer.ai/articles/evaluating-ai-agents-in-ci", "markdown": "https://changegamer.ai/articles/evaluating-ai-agents-in-ci.md", "json": "https://changegamer.ai/api/articles/evaluating-ai-agents-in-ci.json" }, { "slug": "agent-rollout-and-rollback", "title": "How to Roll Out a New AI Agent Version Safely", "description": "An operator playbook for shipping a new agent version without breaking production: in-repo vs. registry prompt storage, a version-numbering comparison, the six-step promotion flow, A/B-test mechanics, the composite-version trace fields, and a rollback drill.", "kind": "sub", "order": 7, "html": "https://changegamer.ai/articles/agent-rollout-and-rollback", "markdown": "https://changegamer.ai/articles/agent-rollout-and-rollback.md", "json": "https://changegamer.ai/api/articles/agent-rollout-and-rollback.json" }, { "slug": "agent-incident-response-runbooks", "title": "How to Build an Incident Response Runbook for AI Agent Failures", "description": "An operator playbook for the moment an AI agent fails in production: a triage step to classify the failure fast, trace freezing before rollback destroys the evidence, first-response depth on the four failure classes, and a blameless post-mortem that feeds back into guardrails and eval.", "kind": "sub", "order": 8, "html": "https://changegamer.ai/articles/agent-incident-response-runbooks", "markdown": "https://changegamer.ai/articles/agent-incident-response-runbooks.md", "json": "https://changegamer.ai/api/articles/agent-incident-response-runbooks.json" }, { "slug": "timeouts-and-deadlines-for-ai-agents", "title": "How to Set Timeouts for AI Agent Tool Calls", "description": "A deep-dive on timeout and deadline design for AI agents: sizing LLM-call, tool-call, and sub-agent-hop timeouts differently, allocating a wall-clock budget across a multi-step chain, and propagating a remaining-deadline value from parent to child calls.", "kind": "sub", "order": 9, "html": "https://changegamer.ai/articles/timeouts-and-deadlines-for-ai-agents", "markdown": "https://changegamer.ai/articles/timeouts-and-deadlines-for-ai-agents.md", "json": "https://changegamer.ai/api/articles/timeouts-and-deadlines-for-ai-agents.json" }, { "slug": "circuit-breakers-and-degraded-mode-for-ai-agents", "title": "How to Design a Circuit Breaker for AI Agents", "description": "A deep-dive on the circuit breaker pattern for AI agents: the Closed/Open/Half-Open state machine with a worked open-source example, where to place a breaker in an agent's call path, and degraded-mode fallback design as its own discipline per dependency type.", "kind": "sub", "order": 10, "html": "https://changegamer.ai/articles/circuit-breakers-and-degraded-mode-for-ai-agents", "markdown": "https://changegamer.ai/articles/circuit-breakers-and-degraded-mode-for-ai-agents.md", "json": "https://changegamer.ai/api/articles/circuit-breakers-and-degraded-mode-for-ai-agents.json" }, { "slug": "agent-observability-for-reliability", "title": "What Should an AI Agent's Observability System Capture?", "description": "An operator playbook for instrumenting an AI agent: the trace/span model behind a run, the OpenTelemetry GenAI attributes that name each field, the fields worth capturing per span, and what to redact before any of it gets logged.", "kind": "sub", "order": 11, "html": "https://changegamer.ai/articles/agent-observability-for-reliability", "markdown": "https://changegamer.ai/articles/agent-observability-for-reliability.md", "json": "https://changegamer.ai/api/articles/agent-observability-for-reliability.json" }, { "slug": "agent-reliability-production-checklist", "title": "The AI Agent Production Reliability Checklist", "description": "A go/no-go checklist that turns the agent reliability pillar's twelve-discipline closing list into checkable gates — the specific artifact, header, or trace field that proves each one holds, with a link to whichever sibling article owns its mechanics.", "kind": "sub", "order": 12, "html": "https://changegamer.ai/articles/agent-reliability-production-checklist", "markdown": "https://changegamer.ai/articles/agent-reliability-production-checklist.md", "json": "https://changegamer.ai/api/articles/agent-reliability-production-checklist.json" } ] }, "navigation": { "pillar": null, "previous": null, "next": null }, "resources": [ { "slug": "reliable-tool-calling", "html": "https://changegamer.ai/resources/reliable-tool-calling", "markdown": "https://changegamer.ai/resources/reliable-tool-calling.md", "json": "https://changegamer.ai/api/resources/reliable-tool-calling.json" }, { "slug": "structured-outputs-and-json-mode", "html": "https://changegamer.ai/resources/structured-outputs-and-json-mode", "markdown": "https://changegamer.ai/resources/structured-outputs-and-json-mode.md", "json": "https://changegamer.ai/api/resources/structured-outputs-and-json-mode.json" }, { "slug": "handling-rate-limits-and-retries", "html": "https://changegamer.ai/resources/handling-rate-limits-and-retries", "markdown": "https://changegamer.ai/resources/handling-rate-limits-and-retries.md", "json": "https://changegamer.ai/api/resources/handling-rate-limits-and-retries.json" }, { "slug": "durable-execution-for-agents", "html": "https://changegamer.ai/resources/durable-execution-for-agents", "markdown": "https://changegamer.ai/resources/durable-execution-for-agents.md", "json": "https://changegamer.ai/api/resources/durable-execution-for-agents.json" }, { "slug": "agent-guardrails", "html": "https://changegamer.ai/resources/agent-guardrails", "markdown": "https://changegamer.ai/resources/agent-guardrails.md", "json": "https://changegamer.ai/api/resources/agent-guardrails.json" }, { "slug": "evaluating-ai-agents", "html": "https://changegamer.ai/resources/evaluating-ai-agents", "markdown": "https://changegamer.ai/resources/evaluating-ai-agents.md", "json": "https://changegamer.ai/api/resources/evaluating-ai-agents.json" }, { "slug": "testing-ai-agents", "html": "https://changegamer.ai/resources/testing-ai-agents", "markdown": "https://changegamer.ai/resources/testing-ai-agents.md", "json": "https://changegamer.ai/api/resources/testing-ai-agents.json" }, { "slug": "agent-observability", "html": "https://changegamer.ai/resources/agent-observability", "markdown": "https://changegamer.ai/resources/agent-observability.md", "json": "https://changegamer.ai/api/resources/agent-observability.json" }, { "slug": "shipping-agents-to-production", "html": "https://changegamer.ai/resources/shipping-agents-to-production", "markdown": "https://changegamer.ai/resources/shipping-agents-to-production.md", "json": "https://changegamer.ai/api/resources/shipping-agents-to-production.json" } ] }