Prompt Caching for AI Agents
Cross-provider prompt caching reference: how to activate it, minimum token thresholds, TTLs, read-vs-write pricing, and when it pays off for agentic workloads.
Prompt caching lets the provider reuse the KV-cache from a prior request when the leading prefix of a new request matches one it has already processed. Repeated tokens are read at a fraction of the standard input price instead of being recomputed. For agents that send the same system prompt, tool definitions, or reference documents on every loop iteration, caching is the single highest-ROI cost lever available. See /resources/agent-cost-latency-optimization for the broader optimization stack.
When does caching pay off?
Caching pays off when:
- A large, stable block (system prompt, tool schemas, RAG chunks, few-shot examples) repeats across many requests within the TTL window.
- The stable prefix exceeds the provider minimum (roughly 1,024–2,048 tokens depending on provider and model).
- Requests arrive frequently enough that write cost is amortized over multiple cache reads within the TTL.
- Agent loop step count is high — savings compound linearly with the number of calls.
Caching does not pay off when:
- Every request has a different prefix (no stable leading block).
- Request volume is very low — a single cache read may not recoup the write premium.
- The TTL expires between requests (e.g., batch jobs spaced hours apart).
- Output tokens dominate cost — caching only discounts input tokens.
How to trigger caching (by provider)
Anthropic (Claude) — manual activation. Place a cache_control field on a content block to mark the cache breakpoint (up to 4 breakpoints per request):
{ "type": "text", "text": "...long stable context...",
"cache_control": { "type": "ephemeral" } }
The default TTL is 5 minutes; a 1-hour TTL is available via "ttl": "1h". Confirm a cache hit by reading cache_creation_input_tokens and cache_read_input_tokens in the response usage object. Cache reads are billed at a large discount versus base input; cache writes carry a modest premium (higher for the 1-hour TTL). Verify the current multipliers and per-model minimum cacheable tokens in the pricing docs.
OpenAI (GPT-4o and later) — fully automatic, no code change. Caching activates when the prompt is 1,024 tokens or longer; the API caches the longest matching prefix in 128-token increments. Cached counts appear in usage.prompt_tokens_details.cached_tokens.
Google Gemini — two modes. Implicit caching (Gemini 2.5+) is automatic with no code, with a minimum of roughly 2,048 tokens. Explicit context caching creates a named cache object with a configurable TTL (default ~60 minutes) referenced by ID; it adds a storage cost per cached-token-hour. Use explicit caching when you need guaranteed reuse.
DeepSeek — automatic prefix caching applied when requests share a common prefix from token 0 (no partial mid-input matches). Cache-hit input is priced far below cache-miss; exact thresholds and rates are not consistently documented — verify in your billing dashboard.
Cross-provider comparison
| Provider | Activation | Min tokens | TTL | Read discount | Notes |
|---|---|---|---|---|---|
| Anthropic Claude | Manual (cache_control breakpoints) |
~512–4,096 (model-dependent) | 5 min default, 1 h opt-in | Large (cache read ≈ 0.1× base input) | Write carries a premium (higher for 1 h); up to 4 breakpoints |
| OpenAI GPT-4o+ | Automatic (prefix-based) | 1,024 | ~5–10 min idle (longer for some models) | Significant on cached prefix (verify per model) | Caches in 128-token increments; no explicit controls |
| Google Gemini 2.5+ | Implicit (auto) or explicit (named cache) | ~2,048 | Implicit: opportunistic; Explicit: ~60 min, configurable | Large on cached read | Explicit caching adds a per-cached-token-hour storage cost |
| DeepSeek | Automatic (exact prefix from token 0) | Not officially documented | Not officially documented | Large cache-hit discount | Exact-prefix match only; verify current pricing |
Exact discount percentages and minimum-token thresholds change per model and over time — treat this table as directional and confirm against each provider's current pricing docs (linked below) before relying on a figure.
Structuring prompts to maximize cache hits
- Stable content at the top. Put the system prompt, tool schemas, and reference documents before any dynamic content — cache matching is prefix-based.
- Dynamic content at the tail. User messages, current task context, and retrieved chunks go after the cache breakpoint.
- Keep stable blocks stable. Any edit to a cached block invalidates the cache; treat cached content as immutable across a session.
- Monitor hit rates. All major providers expose cache-read token counts in the usage response. Track cache_read_tokens / total_input_tokens per endpoint; a low ratio for an agent with a stable system prompt signals missed opportunities.
For prompt-structure principles that interact with caching, see /resources/prompt-context-engineering.
Stacking with batch discounts
Prompt caching and a batch API (50% off all tokens — see /resources/handling-rate-limits-and-retries) stack independently: a batched request that hits cache pays the batch rate on uncached tokens and the cache-read rate on cached tokens — the cheapest combination for non-latency-sensitive work.
Verified sources
- Anthropic prompt caching docs: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- Anthropic pricing (cache write/read multipliers): https://platform.claude.com/docs/en/about-claude/pricing
- OpenAI prompt caching announcement: https://openai.com/index/api-prompt-caching/
- Google Gemini implicit caching announcement: https://developers.googleblog.com/en/gemini-2-5-models-now-support-implicit-caching/
- Google Gemini context caching docs: https://ai.google.dev/gemini-api/docs/caching
- DeepSeek context caching guide: https://api-docs.deepseek.com/guides/kv_cache