← All guides · Agent reliability in production

How to Design a Circuit Breaker for AI Agents

Part 10 of Agent reliability in production · 1,592 words · ~7 min read · published 2026-08-30 · updated 2026-08-30 · Markdown variant

A deep-dive on the circuit breaker pattern for AI agents: the Closed/Open/Half-Open state machine with a worked open-source example, where to place a breaker in an agent's call path, and degraded-mode fallback design as its own discipline per dependency type.

Part of the Agent Guardrails and the AI Agent Reliability Playbook guide.

The agent reliability in production pillar names circuit breakers and degraded mode in one paragraph and gives two illustrative examples. This article expands both halves of that paragraph into their own disciplines: the circuit breaker's internal state machine, using a real, citable worked example rather than an invented number, and degraded-mode fallback design as a decision made per dependency type before an incident happens, not during one.

What is the circuit breaker pattern for an AI agent?

The circuit breaker pattern is a safeguard that stops an agent from continuing to hammer a dependency that has already demonstrated it is down, converting an open-ended stall into an immediate, predictable failure instead. It works on a longer time horizon than ordinary retry logic: a retry loop assumes the very next attempt still has a real chance of landing, while a circuit breaker assumes the opposite once its threshold is crossed — that the dependency needs a recovery window with no additional load, and that every request sent during that window wastes the caller's time at best and adds strain to an already-struggling service at worst. The mechanics of that earlier retry layer — transient-versus-terminal classification, exponential backoff with full jitter, idempotency keys for a retried side effect — are their own discipline and are not re-derived here; see how to make AI agent retries idempotent for that mechanism in full.

The three-state circuit breaker state machine

A circuit breaker for an agent operates as a state machine with exactly three states — Closed, Open, and Half-Open — moving between them based on the outcomes of the calls it observes. This three-state design is widely attributed to Michael Nygard's book "Release It!" and popularized broadly in software-architecture writing since, though that specific attribution is itself widely repeated rather than independently verified against a primary source this session.

• Closed is the normal operating state: every request is let through to the dependency, and the breaker silently tracks the outcome of each call — success or failure — against its configured threshold.
• Open is the tripped state: once the failure threshold is crossed, the breaker stops sending requests to the dependency entirely and rejects every call immediately — either with an explicit error or a fallback response — without attempting the real downstream call at all, for the duration of a cooldown period.
• Half-Open is the recovery-testing state: once the open-state cooldown timer expires, the breaker lets a small, limited number of probe requests through to check whether the dependency has actually recovered. If those probes succeed, the breaker resets to Closed and resumes normal traffic; if they fail, it returns to Open and starts the cooldown again.

This three-state design is what separates a circuit breaker from a simple "give up after N failures" flag: the Half-Open state gives the system a controlled, low-risk way to detect recovery on its own, rather than requiring a human to notice the dependency is back and manually re-enable traffic to it.

A worked example, from one open-source implementation

No universal failure-threshold number, cooldown duration, or probe count exists for a circuit breaker — each dependency's own failure and latency profile should drive its own choice, and treating any single set of numbers as a default to copy elsewhere is a mistake. As of August 2026, one concrete, citable implementation of the pattern is the AWS CDK Patterns open-source repository's "the-lambda-circuit-breaker" example, which configures a failureThreshold of 3 consecutive failures before the breaker opens, a successThreshold of 2 consecutive successful probes before it closes again from Half-Open, and a 10-second per-call timeout, with a dedicated fallback function invoked in place of the real call while the breaker sits Open. Treat these three numbers as one illustrative example from a single implementation, not a prescription: a dependency behind a slow, expensive downstream call might reasonably open after a single failure, while a cheap, flaky one might tolerate ten before tripping — the right values come from that dependency's own observed behavior, not from a table of defaults.

Where should a circuit breaker sit in an agent's call path?

A circuit breaker belongs wrapped tightly around the specific call to one dependency — one tool endpoint, one model provider, one retrieval index — rather than wrapped around an entire multi-step agent run, because its failure count only means something if it tracks one dependency's behavior in isolation. A breaker that blends failures from several unrelated calls into one counter trips for the wrong reason and recovers at the wrong time: a spike of failures from one flaky tool would needlessly cut off traffic to an unrelated, healthy model provider sharing the same counter. Each dependency an agent calls — each distinct tool, each model provider, each retrieval backend — gets its own breaker instance with its own threshold, cooldown, and state, sized to that dependency's own latency and failure characteristics rather than a value shared across the whole system.

Why does degraded mode have to be designed before an incident, not during one?

Degraded mode has to be designed before an incident because deciding, for the first time, what an agent should say or do when a dependency is unreachable is a design decision, not a triage action, and an on-call responder mid-incident has neither the time nor the full context to make that decision safely under pressure. Degraded mode is the specific fallback behavior a tripped circuit breaker routes traffic into: instead of the caller simply receiving an error, the agent serves a pre-defined, reduced-but-usable response. Designing it in advance means answering, per dependency, how much quality reduction is acceptable and how it gets disclosed before that dependency ever actually fails — a decision with product, correctness, and disclosure implications that belong in a design review, not in a Slack thread during an outage.

Degraded-mode design by dependency type

The table below works through a reasonable fallback for three dependency types an agent commonly relies on — one grounded in a documented, corpus-verified mechanism, and two reasoned from how those systems are architected rather than from an observed or documented practice:

The pattern across all three: each fallback keeps the agent answering rather than failing outright, and each one accepts a specific, bounded quality reduction instead of an unbounded one — a stale cache instead of nothing, an unranked list instead of nothing, a slower or different model instead of nothing.

How does a circuit breaker fit alongside retries and incident response?

A circuit breaker sits between an agent's retry logic and its incident-response process, handling the layer neither of the other two owns. Retry-and-backoff logic decides what to do about a single failing call and is not re-derived here — see how to make AI agent retries idempotent for that mechanism. Incident-response runbooks decide what a team does once an outage is confirmed, naming "trip a circuit breaker so the agent switches to its pre-defined degraded mode" as the first response to a tool or dependency outage without covering the breaker's own internal mechanics — see how to build an incident response runbook for AI agent failures for that triage and response layer. This article is the layer in between: the state machine that decides, mechanically and without human intervention, when to stop calling a dependency and when to try it again. The full ten-dimension production ship gate this discipline sits inside, including cost controls and rollback, is in shipping AI agents to production.

Where this leaves you

Build one circuit breaker per dependency — not one per agent run — sized with a failure threshold, a cooldown duration, and a half-open probe count drawn from that specific dependency's own failure and latency profile rather than copied from any single example, worked or otherwise. Design degraded mode for each dependency type ahead of time, deciding what a reduced-but-honest response looks like before the dependency ever actually fails, and disclose the reduction rather than hiding it. For the other eleven reliability disciplines this one sits inside, see the agent reliability in production pillar.

#agents #reliability #circuit-breaker #degraded-mode #production #resilience