How to Make AI Agent Retries Idempotent
A deep-dive on retrying agent tool calls safely: the transient-vs-terminal decision, why an agent side effect can fire before a failure signal reaches the caller, idempotency-key mechanics (run ID + step index), the unknown-outcome edge case, and where idempotency keys do not reach.
- Exponential backoff with full jitter is the standard retry pattern for agent tool calls — it randomizes the wait between zero and an exponentially growing cap while always treating a provider's Retry-After header as a hard floor, as documented in the handling-rate-limits-and-retries reference as of July 2026.
- An agent tool call can trigger a real side effect — a payment, an email, a database write — before a timeout or a dropped connection ever reaches the calling code, which makes "no response received" a fundamentally different outcome from "the call failed cleanly."
- A safe idempotency key for an agent step combines the workflow run's own identity with that step's position inside the run — never a clock reading or a freshly minted UUID, since either one looks different on every attempt and so cannot tell a downstream service "this is the same request as before."
- Retrying the same step after a crash or a dropped connection must reuse that step's original idempotency key so the duplicate attempt collapses into a no-op, while a genuinely new run of the same workflow needs a fresh run identity and therefore a fresh key.
- Idempotency-key support is not a guarantee of the HTTP or REST tradition — it is a feature an individual API chooses to implement or not — so an agent cannot assume a downstream service has one and needs an application-level dedupe record ready as a fallback.
The agent reliability in production pillar surveys twelve reliability disciplines at survey depth; this article expands two of them into an operator playbook — how an agent decides what to retry, and how it makes a retry safe once a side effect might already have happened before any failure signal came back.
How should an agent retry a failed tool call?
An agent should retry a failed tool call only after classifying the failure as transient, then space out attempts on a jittered exponential schedule that never dips below whatever wait time the provider explicitly mandates. A 429 (rate limited), a 5xx server error, and a plain network timeout are transient — the condition that caused them is temporary, and a second attempt has a real chance of succeeding. A 400 (malformed request) or a 401/403 (bad credentials or missing permission) is terminal — the request itself is wrong, and retrying it produces the identical failure every time while burning a retry budget that should instead escalate immediately to a human or a fallback path. The full backoff formula, the provider-by-provider rate-limit header reference (x-ratelimit-remaining-tokens, anthropic-ratelimit-requests-remaining, and their peers), and the RPM/TPM/TPD tier structure that governs how much headroom an agent actually has live in handling rate limits and retries — this article assumes that mechanism and focuses on what changes once a retried call can duplicate a side effect, which the next three sections cover.
One detail worth calling out at fleet scale: an agent deployment rarely retries in isolation. Dozens or hundreds of concurrent agent runs can hit the same rate limit within the same window — after a shared model deployment briefly degrades, or after a deploy restarts many workflow replicas at once — and if every one of them backs off on the same fixed schedule, they collide again at the next retry round instead of spreading out. Full jitter, not a fixed delay or a bare exponential schedule with no randomization, is what keeps a fleet of independently retrying agent runs from synchronizing into a self-inflicted retry storm against the same endpoint.
Why are agent retries a harder problem than a typical API client's retries?
Agent retries are harder than a typical stateless API client's retries because an agent's tool call frequently triggers a real-world side effect, and that side effect can complete on the far side before any failure signal — a timeout, a dropped connection, a crashed process — ever reaches the code deciding whether to retry. A conventional REST client calling a read-only endpoint can safely assume "I got no response, so nothing happened" and simply resend the request. An agent issuing a tool call that charges a card, sends an email, or writes a database row cannot make that assumption: the request may have arrived, executed, and only the response was lost. Naively resending it does not repeat "nothing happened" — it repeats the side effect itself, producing a duplicate charge, a duplicate email, or a corrupted counter.
This gap is exactly why idempotency has to be designed in deliberately rather than assumed. If a process dies partway through a side-effecting call, the next attempt — whether triggered by your own backoff loop or by a durable-execution engine replaying its checkpoint log after a restart — has no way to know whether that call already landed, and resending it blind risks repeating the exact effect it was trying to complete. The fix is not a smarter retry loop; it is a key that lets the downstream service recognize the second attempt as identical to the first.
How do you build an idempotency key that actually deduplicates a retry?
Build an idempotency key from two stable identifiers: the workflow run's own identity, and the index of the specific step being attempted inside that run. A clock reading and a freshly minted UUID are both disqualified by the same property — each is guaranteed to come out different on the next attempt — so a key built from either one shifts every time it is generated, and a key that shifts on retry cannot signal "same request" to anything downstream.
def step_idempotency_key(run_id: str, step_index: int) -> str:
# Stable across every retry of THIS step, in THIS run.
# A crash, a 5xx, or a dropped connection can all trigger a retry
# that reuses this exact key — that is the point.
return f"{run_id}:step-{step_index}"
def charge_customer(run_id, step_index, amount, customer_id):
key = step_idempotency_key(run_id, step_index)
return payment_api.charge(
amount=amount,
customer_id=customer_id,
idempotency_key=key, # downstream service treats a repeat key as a no-op
)
Pass that key to whatever field the downstream service designates for deduplication — a payment processor, a transactional-email sender, or a messaging platform typically has one for exactly this reason — so a duplicate attempt collapses into a no-op response instead of a second real effect. This is the same mechanism durable execution engines rely on internally: resuming a workflow after a crash replays its orchestration logic from the recorded log, and any step still in progress when the crash happened can end up scheduled again, so the engine (or the code layered on top of it) needs a stable key to make that replay land as a no-op, as covered in more depth in durable execution for long-running agents — idempotency keys are foundational to how durable execution avoids duplicating side effects on replay, though the mechanics of replay and checkpointing themselves are that resource's and a future sub-article's territory, not this one's.
Same-step retry vs. a legitimate new run
A retry of the same step must reuse that step's original key; a new run of the same workflow must not. Building the key from both the run's own identifier and the step's position inside it makes this fall out automatically: retrying step 3 of run abc123 after a timeout produces the identical key abc123:step-3 every time, so the downstream service correctly treats every attempt as one logical charge. Starting a fresh run of the same workflow — a different customer, a different day, a legitimately new request — gets its own new identifier, and therefore its own new key, so it is never mistaken for a duplicate of unrelated prior work. The failure mode to watch for is the reverse: a key built only from the step's position with no run identity folded in would collide across every run that happens to reach "step 3," silently blocking legitimate charges that have nothing to do with each other.
What do you do when a call's outcome is genuinely unknown?
When a call's outcome is genuinely unknown — a timeout, a dropped connection, a proxy that hung up mid-request — treat it as "may have executed" and retry with the same idempotency key, never as a clean failure that is safe to reattempt without one. This is the distinction that actually drives what to do next: a clean error response (a 400, a 401, a well-formed 429) tells you unambiguously what happened on the server side, but a timeout or a connection drop tells you nothing — the request could have failed before it was ever processed, or it could have completed in full with only the acknowledgment lost in transit. Resending with the same key is what makes that ambiguity safe to live with: if the first attempt never landed, the retry executes normally; if it already landed, the downstream service's own deduplication turns the retry into a no-op — either way, the caller does not have to resolve the ambiguity itself before deciding to resend.
| Failure signal | What it tells you | Needs an idempotency key |
|---|---|---|
| 400 / validation error | Clean, terminal — rejected before any side effect ran | No — fix the request, don't retry |
| 401 / 403 | Clean, terminal — credentials or permissions, not payload | No — escalate, don't retry |
| 429 rate limited | Clean, transient — provider explicitly did not execute it | No for the retry decision, yes if the call is side-effecting regardless |
| Clean 5xx response | Ambiguous — some 5xx mean "never ran," some mean "ran, then failed to reply" | Yes, for any side-effecting call |
| Timeout / dropped connection | Unknown — request may have reached and executed before the response was lost | Yes, always |
| Process crash mid-step | Unknown — the step may have been mid-flight when the crash occurred | Yes, always |
Where idempotency keys don't reach
Idempotency keys are not a universal solve, because nothing in the HTTP or REST tradition requires an API to support one — it is a feature each service chooses to implement or not, and a service that never opted in has no idempotency-key parameter to pass a key to at all. The pattern described above only works where the receiving system was built to support it. Where it isn't supported, the fallback is an application-level dedupe record: before attempting a side-effecting call, check your own store for a record that the same run-and-step key already succeeded, and only proceed if it hasn't. This shifts the deduplication responsibility from the downstream service to your own system, which is strictly more work to build and maintain correctly than passing a header — and for a call where neither an idempotency-key header nor a reliable own-side dedupe record is feasible (a one-off webhook to a system you don't control, for instance), the honest answer is that the duplication risk cannot be fully eliminated, only reduced by minimizing the retry window and alerting on any retry of that specific call so a human can check for a duplicate after the fact.
Where this leaves you
Classify every tool-call failure as transient or terminal before deciding whether to retry, back off with full jitter so a fleet of agents doesn't resynchronize into its own retry storm, and never let "no response" be treated as "nothing happened" for a side-effecting call. Build every idempotency key from the run's identity and the step's position within it, reuse that exact key on every retry of that step, and generate a new key only when the run itself is genuinely new. Where a downstream API has no idempotency-key support, fall back to an application-level dedupe record rather than assuming the risk away. For the ten other reliability disciplines this one sits inside, see the agent reliability in production pillar; for how this same key pattern underwrites crash-recovery replay in a durable workflow, see durable execution for long-running agents.
Frequently asked questions
- What should an idempotency key for an agent tool call be derived from?
- An idempotency key should combine the workflow run's own identity with the specific step's position inside that run. A clock reading or a newly generated UUID cannot serve this purpose, because each one differs on every attempt by construction, and only a value that stays constant across retries lets a downstream service recognize a repeat as the same request rather than a brand-new one.
- Is it safe to retry a tool call after a timeout with no response?
- A tool call that timed out with no response is not automatically safe to retry as if nothing happened, because the request may have already reached the server and executed before the connection dropped, so any side-effecting call in this state needs to be resent with the same idempotency key it used on the first attempt, not treated as a clean do-over.
- Do all APIs support idempotency keys for deduplicating retries?
- No — idempotency-key support is not part of the HTTP or REST specification, it is a feature an individual API chooses to implement or not, so an agent cannot assume every downstream service has one. Calling a system that lacks that support means falling back to an application-level dedupe record — checking whether the effect already happened before attempting it again — or accepting the duplication risk for that specific integration.
- What is the difference between a transient and a terminal failure for retry purposes?
- A transient failure such as a 429 rate limit, a 5xx server error, or a network timeout is one where retrying with backoff has a real chance of succeeding because the underlying condition is temporary, while a terminal failure such as a 400 bad request or a 401 authentication error will fail identically on every retry because the problem is in the request itself, so retrying it only delays the inevitable escalation.
This guide is free and stays free. The reference corpus behind it — machine-readable contracts, verified primary sources, continuously refreshed — is the paid product: a €5 starter key unlocks every premium reference for one agent via API; a €25 corpus license delivers the full corpus as RAG / fine-tuning data with an explicit AI-use grant; the €150 enterprise license adds commercial redistribution rights.