# Handling LLM Rate Limits (HTTP 429) and Retries for Agents

> A practical reference for agent builders: what a 429 means, how to read provider rate-limit headers, exponential backoff with jitter, client-side throttling, and when to use a batch API.

Category: Guide · Updated: 2026-07-15 · Tags: rate-limits, http-429, retry, backoff, throttling, openai, anthropic, agents, reliability
Canonical: https://changegamer.ai/resources/handling-rate-limits-and-retries
Variants: [HTML](https://changegamer.ai/resources/handling-rate-limits-and-retries) · [JSON](https://changegamer.ai/api/resources/handling-rate-limits-and-retries.json)
License: https://changegamer.ai/license.xml · Access: free

When an LLM provider returns HTTP 429 (Too Many Requests), the calling agent has exceeded one or more rate limits for the current window. **The first thing to do: read the `Retry-After` header (or its provider-specific equivalent) and wait that many seconds before retrying. Do not retry immediately.** Most providers also return per-dimension headers on every successful response, letting you throttle proactively rather than reactively.

## Key facts

- On a 429, the immediate move is to honor the `Retry-After` (or equivalent) header instead of retrying right away, and providers also expose per-dimension headroom on every success so agents can throttle before ever hitting a limit.
- Limits actually stack across at least three windows — raw request-per-minute counts, combined or separately tracked input/output token-per-minute budgets, and a daily ceiling that overrides any remaining per-minute headroom.
- Both providers loosen these ceilings as usage and spend grow, through numbered tiers that advance on account age/cumulative spend or purchased credit — but exact thresholds shift over time, so the practical move is to check the dashboard and request headroom ahead of need rather than react to production errors.
- Randomizing retry delay across the full range up to an exponentially growing cap — while never going below whatever wait time the provider mandated — keeps a fleet of retrying clients from all hammering the API at once.
- Before ever reaching a 429, agents can self-throttle by tracking a local budget from the remaining-headroom headers, capping concurrent in-flight calls, or pre-estimating token cost to hold back requests that would blow the budget.
- Because it draws from a separate quota pool and runs at half the per-token price with same-day turnaround, the batch API suits offline, non-interactive workloads like evals or bulk summarization rather than latency-sensitive agent calls.

## What the rate-limit headers tell you

| Header | OpenAI | Anthropic |
|---|---|---|
| Requests remaining | `x-ratelimit-remaining-requests` | `anthropic-ratelimit-requests-remaining` |
| Tokens remaining | `x-ratelimit-remaining-tokens` | `anthropic-ratelimit-tokens-remaining` |
| Input/output tokens remaining | — | `anthropic-ratelimit-input-tokens-remaining` / `...-output-tokens-remaining` |
| Limit (requests / tokens) | `x-ratelimit-limit-requests` / `...-tokens` | `anthropic-ratelimit-requests-limit` / `...-tokens-limit` |
| Reset time | `x-ratelimit-reset-requests` / `...-tokens` | `anthropic-ratelimit-requests-reset` / `...-tokens-reset` (RFC 3339) |
| Wait before retry | `Retry-After` (on 429) | `retry-after` (on 429) |

Google Gemini exposes `x-ratelimit-remaining` and `retry-after` on the same pattern. Anthropic tracks input and output tokens in separate ITPM and OTPM limits; cached input tokens (cache reads) generally do not count toward ITPM, so prompt caching raises effective throughput without a tier change (see /resources/prompt-caching-for-agents).

## RPM, TPM, and TPD — what each limit covers

**RPM (requests per minute)** — total HTTP calls regardless of payload size; high-frequency, low-token bursts hit this first. The window is typically a rolling 60 seconds.

**TPM (tokens per minute)** — input + output tokens per rolling minute. Long contexts or large system prompts burn TPM quickly even at low RPM. Anthropic enforces separate ITPM and OTPM; OpenAI combines them.

**TPD / RPD (per day)** — a daily ceiling across all per-minute windows; hitting it means waiting for the reset regardless of per-minute headroom. Often the binding constraint for lower tiers running overnight workloads.

## Usage tiers and how limits advance

Both major providers raise your limits as you spend more. OpenAI starts every new project on a Free usage tier (low limits, some models unsupported) and uses numbered usage tiers (Tier 1–5 above it) that advance automatically as cumulative spend and account age cross thresholds; Anthropic uses numbered tiers gated by cumulative credit purchase plus a monthly-invoicing tier with negotiated limits. The exact spend thresholds and per-model RPM/TPM/ITPM numbers change over time and per model — read them from your provider dashboard or the official rate-limit docs rather than hard-coding them. The practical rule at every tier: design for the limit you have today, and request an increase before you need it, not after 429s appear in production.

## Exponential backoff with full jitter

Full jitter — randomizing the delay uniformly between zero and the exponential ceiling — spreads retrying agents across time and prevents synchronized retry storms. Always enforce the provider's `Retry-After` value as a hard floor.

```
MAX_RETRIES = 7
BASE_DELAY  = 1      # seconds
CAP_DELAY   = 60     # seconds

def call_with_backoff(request):
    for attempt in range(MAX_RETRIES):
        response = send(request)
        if response.status != 429:
            return response
        floor = int(response.headers.get("Retry-After", 0))   # provider-mandated floor
        ceiling = min(CAP_DELAY, BASE_DELAY * 2 ** attempt)
        wait = max(floor, random_uniform(0, ceiling))         # full jitter
        sleep(wait)
    raise RateLimitExceeded("max retries reached")
```

The ceiling grows exponentially (1s, 2s, 4s, 8s…) but is capped; the `Retry-After` floor ensures you never retry before the provider permits. Source: aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/.

## Throttling client-side before you hit 429

- **Token bucket** — maintain a local token count from the `*-remaining` headers; subtract estimated cost before each request and sleep until the reset timestamp if it would go negative.
- **Concurrency cap** — bound in-flight requests to a fraction of RPM with a semaphore or async queue.
- **Estimated token pre-check** — tokenize the prompt before sending and queue requests that would exceed remaining headroom.

## When to use a batch API instead

For non-urgent, non-interactive work, both major providers offer a batch API that runs on a separate rate-limit pool (so it does not consume your standard RPM/TPM) and costs **50% less per token**, with results delivered within 24 hours:

- **OpenAI Batch API** — submit a JSONL file of many requests; 50% discount; separate pool. Source: platform.openai.com/docs/guides/batch.
- **Anthropic Message Batches API** — submit up to 100,000 requests per batch; 50% off input and output. Source: platform.claude.com/docs/en/api/rate-limits.

Use a batch API for evals, embedding large corpora, offline summarization, and synthetic-data generation — not for interactive agents that need a result in seconds.

## Provider fallback via a gateway

When a provider is at capacity, an LLM gateway can route to a secondary provider instead of retrying the same endpoint — useful for daily-limit exhaustion or regional outages. See /resources/ai-gateways-llm-routing. For workflow-level retries and idempotency, see /resources/durable-execution-for-agents; for malformed-output retries, see /resources/reliable-tool-calling.

## Verified sources

- OpenAI rate limits guide: https://platform.openai.com/docs/guides/rate-limits
- OpenAI Batch API guide: https://platform.openai.com/docs/guides/batch
- Anthropic rate limits reference: https://platform.claude.com/docs/en/api/rate-limits
- AWS: Exponential Backoff and Jitter: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/

---

## Related resources

- [Computer Use and Browser Automation for Agents](https://changegamer.ai/resources/computer-use-browser-automation.md): Two-layer reference: vendor computer-use APIs (Anthropic, OpenAI CUA, Google Gemini) that translate screenshots to actions, and the open harnesses (Playwright MCP, browser-use, Stagehand, Skyvern) that execute those actions — with loop mechanics, reliability tradeoffs, and security gates.
- [Prompt Caching for AI Agents](https://changegamer.ai/resources/prompt-caching-for-agents.md): Cross-provider prompt caching reference: how to activate it, minimum token thresholds, TTLs, read-vs-write pricing, and when it pays off for agentic workloads.
- [Streaming Responses for Agents](https://changegamer.ai/resources/streaming-for-agents.md): Transport formats, provider event schemas, and practical concerns for consuming streamed LLM responses in production agents: SSE mechanics, OpenAI (Chat Completions and Responses API) and Anthropic event formats, partial-JSON tool-call parsing, backpressure, cancellation, and gateway proxying.
- [Durable Execution for Long-Running Agents](https://changegamer.ai/resources/durable-execution-for-agents.md): Vendor-neutral reference on durable execution: event logs, replay determinism, idempotency, retries, and human-in-the-loop pause/resume — plus a cross-vendor survey and tradeoffs guide for Temporal, Restate, DBOS, Inngest, Step Functions, Azure Durable Functions, Cloudflare Workflows, GCP Workflows, LangGraph, and OpenAI Agents SDK.

---

Index of all resources: https://changegamer.ai/llms.txt · Full corpus: https://changegamer.ai/llms-full.txt · Corpus data (NDJSON): https://changegamer.ai/api/corpus.jsonl · Offers: https://changegamer.ai/api/pricing.json