#reliability

5 resources and 16 guides tagged #reliability on ChangeGamer.

Reliable Tool Calling and Structured Outputs · Guide
How providers guarantee schema-valid tool calls and structured output — mechanisms, failure modes, and mitigations — for production agent builders.
Durable Execution for Long-Running Agents · Guide
Vendor-neutral reference on durable execution: event logs, replay determinism, idempotency, retries, and human-in-the-loop pause/resume — plus a cross-vendor survey and tradeoffs guide for Temporal, Restate, DBOS, Inngest, Step Functions, Azure Durable Functions, Cloudflare Workflows, GCP Workflows, LangGraph, and OpenAI Agents SDK.
Handling LLM Rate Limits (HTTP 429) and Retries for Agents · Guide
A practical reference for agent builders: what a 429 means, how to read provider rate-limit headers, exponential backoff with jitter, client-side throttling, and when to use a batch API.
Shipping AI Agents to Production: A Production-Readiness Checklist · Guide
End-to-end checklist for productionizing an AI agent — evaluation gates, observability, guardrails, cost controls, resilience, durability, HITL approvals, secrets, rollback, and incident response.
LLM Model Deprecation: Detecting and Handling End-of-Life Models · Reference
How the general IETF Sunset/Deprecation HTTP headers work, why none of the three major LLM APIs actually send them, and each vendor's real notice periods and retirement mechanics — plus detection and fallback patterns for agents that pin a model ID today.

Guides

Common MCP Server Failure Modes and How to Fix Them · guide
A runtime playbook for the two MCP server failure modes with no dedicated deep-dive elsewhere: unrecoverable state after a mid-call crash, and malformed or hallucinated tool calls that reach the handler despite upstream validation.
Common RAG Failure Modes and How to Fix Them · guide
An operator runbook for the four RAG failure classes with no dedicated deep-dive elsewhere: retrieval miss, context overload, injection via content at ingestion time, and silent quality degradation — symptom, first diagnostic, and fix for each.
Agent Guardrails and the AI Agent Reliability Playbook · pillar guide
Agent guardrails plus the eleven other disciplines that make an AI agent reliable in production: tool calling, retries, durable execution and rollout.
How to Make AI Agent Tool Calling Reliable · guide
An operator playbook for tool-calling contracts: per-provider strict-mode config (OpenAI, Anthropic, Gemini, self-hosted grammars), the full failure-mode-to-fix table, what BFCL actually measures, and how to stop parallel tool calls from breaking a dependency chain.
Structured Outputs vs Tool Calling: When to Use Each · guide
A decision framework for structured outputs versus tool calling in AI agents, with runnable JSON Schema examples for OpenAI, Anthropic, Gemini, vLLM/SGLang, and llama.cpp, plus mitigation code for truncation, refusal, and grammar-compilation latency.
How to Make AI Agent Retries Idempotent · guide
A deep-dive on retrying agent tool calls safely: the transient-vs-terminal decision, why an agent side effect can fire before a failure signal reaches the caller, idempotency-key mechanics (run ID + step index), the unknown-outcome edge case, and where idempotency keys do not reach.
When Do AI Agents Need Durable Execution? · guide
A deep-dive on durable execution for AI agents: the persisted event log, the replay-determinism constraint, the four architectural shapes mapped across ten engines and frameworks, and a decision framework for when a durable execution engine is worth adding at all.
How to Design Guardrails for AI Agent Reliability · guide
An operator playbook for reliability guardrails: the three checkpoints (input, output, action), layering cheap checks under slow ones with a fail-closed default, the two-of-three-properties rule for when a tool call needs human approval, and logging every verdict against the run trace ID.
How to Evaluate AI Agents in CI · guide
An operator playbook for gating an AI agent release in CI: why agent eval needs trajectory-level scoring across the tasks it actually runs, how public benchmarks diverge as proxies, ground-truth vs LLM-as-judge tool-call scoring, and the three-layer test pyramid that keeps CI fast and non-flaky.
How to Roll Out a New AI Agent Version Safely · guide
An operator playbook for shipping a new agent version without breaking production: in-repo vs. registry prompt storage, a version-numbering comparison, the six-step promotion flow, A/B-test mechanics, the composite-version trace fields, and a rollback drill.
How to Build an Incident Response Runbook for AI Agent Failures · guide
An operator playbook for the moment an AI agent fails in production: a triage step to classify the failure fast, trace freezing before rollback destroys the evidence, first-response depth on the four failure classes, and a blameless post-mortem that feeds back into guardrails and eval.
How to Set Timeouts for AI Agent Tool Calls · guide
A deep-dive on timeout and deadline design for AI agents: sizing LLM-call, tool-call, and sub-agent-hop timeouts differently, allocating a wall-clock budget across a multi-step chain, and propagating a remaining-deadline value from parent to child calls.
How to Design a Circuit Breaker for AI Agents · guide
A deep-dive on the circuit breaker pattern for AI agents: the Closed/Open/Half-Open state machine with a worked open-source example, where to place a breaker in an agent's call path, and degraded-mode fallback design as its own discipline per dependency type.
What Should an AI Agent's Observability System Capture? · guide
An operator playbook for instrumenting an AI agent: the trace/span model behind a run, the OpenTelemetry GenAI attributes that name each field, the fields worth capturing per span, and what to redact before any of it gets logged.
The AI Agent Production Reliability Checklist · guide
A go/no-go checklist that turns the agent reliability pillar's twelve-discipline closing list into checkable gates — the specific artifact, header, or trace field that proves each one holds, with a link to whichever sibling article owns its mechanics.
How to Keep Trace Data When an AI Agent Crashes Mid-Run · guide
How to keep trace data when an AI agent crashes mid-run: write a small run-start record outside the trace buffer, detect orphans, count them as unknown outcomes, and force-decide on shutdown.