# Context Window Management: Budgets, Compaction and Caching > How to work within finite context windows: budget allocation across instructions, evidence and history; truncation versus summarization versus retrieval offload; prefix stability for prompt caching; and attention-quality caveats of very long contexts. Category: Reference · Updated: 2026-08-23 · Tags: context-window, prompt-engineering, caching, rag, memory, agents Canonical: https://changegamer.ai/resources/context-window-management Variants: [HTML](https://changegamer.ai/resources/context-window-management) · [JSON](https://changegamer.ai/api/resources/context-window-management.json) License: https://changegamer.ai/license.xml · Access: free Every model reads a finite context window, and everything an agent needs — instructions, conversation history, retrieved evidence, tool results — competes for the same space. Context-window management is the discipline of allocating that space deliberately instead of discovering its limits through failures. ## Allocate the budget by role Treat the window as partitioned: stable instructions and tool definitions first (they change rarely), then working memory (task state, plan), then retrieved evidence, then recent history. Fixed allocations per tier make behavior predictable — when evidence grows, history shrinks on a rule rather than on vibes. Teams without explicit budgets get the classic failure mode: a long session quietly pushes the system prompt's constraints out of effective attention and the agent drifts. ## Offloading beats squeezing Three mechanisms move information out of the window without losing it: - **Retrieval**: store knowledge outside the window and fetch relevant slices on demand — RAG is context-window management at corpus scale (see /resources/rag-retrieval-for-agents) - **Summarization and compaction**: periodically compress older history into shorter state summaries; keep the summary faithful to decisions and open threads, not just transcripts - **Structured state files**: persist task progress to an external store the agent reads and writes deliberately, rather than re-narrating it every turn ([agent memory & context](/resources/agent-memory-context)) Truncation — dropping oldest messages — is the fallback, not the strategy: it silently amputates constraints and commitments made earlier in the session. ## Long-context caveats Very large windows invite just-include-everything designs. Two cautions temper that: attention quality across very long contexts degrades non-uniformly — models reliably use information at the beginning and end better than material buried mid-context — and cost scales with length whether or not buried content helps. Effective context engineering keeps relevant material close to where generation happens ([prompt-context-engineering](/resources/prompt-context-engineering)). ## Prefix stability makes caching pay Provider-level prompt caching charges reduced rates for repeated prefixes, which rewards one structural choice: put volatile content last. System prompt, tool schemas and standing instructions form the stable prefix; per-turn user content, retrieved evidence and history append after it. Reordering content between turns invalidates cache prefixes and multiplies cost — [prompt caching for agents](/resources/prompt-caching-for-agents) covers the mechanics, and response-level reuse ([agent response caching](/resources/agent-response-caching)) compounds savings for repeated questions. ## Sizing inputs honestly Estimate tokens before sending: English prose runs roughly four characters per token, so word counts convert to approximate token budgets with a ~1.33 multiplier. Leave headroom below the stated maximum for the output you expect back, since output tokens share the same window. Chunking strategy determines how much evidence a retrieval step must fetch for adequate coverage — tighter, better-targeted chunks mean fewer tokens per answered question (see /resources/chunking-strategies-for-rag). --- ## Related resources - [Agent Memory and Context Management](https://changegamer.ai/resources/agent-memory-context.md): Architecture reference for agent memory: types (working, long-term, episodic, semantic, procedural), context-management techniques (summarization, RAG, sliding windows, prompt caching), storage substrates, and memory frameworks — with security notes and cross-links to related guides. - [Prompt and Context Engineering for Agents](https://changegamer.ai/resources/prompt-context-engineering.md): From crafting a single prompt to managing everything an agent sees across a trajectory: system-prompt design, context-window management, failure modes, and a high-leverage checklist. - [Document Extraction and Parsing for Agents](https://changegamer.ai/resources/document-extraction-for-agents.md): Practitioner reference for the document-ingestion pipeline agents use: parse/OCR, layout/structure extraction, schema-constrained field extraction — with a verified tooling landscape (OSS and cloud). - [Embeddings and Vector Search for Agents](https://changegamer.ai/resources/embeddings-vector-search.md): How to pick an embedding model, understand distance metrics, choose an ANN index type, and operate a vector store reliably in agent retrieval pipelines. --- ## Further reading - [Common RAG Failure Modes and How to Fix Them](https://changegamer.ai/articles/rag-failure-modes-runbook.md): An operator runbook for the four RAG failure classes with no dedicated deep-dive elsewhere: retrieval miss, context overload, injection via content at ingestion time, and silent quality degradation — symptom, first diagnostic, and fix for each. - [GraphRAG vs Vector RAG: When to Use a Knowledge Graph Instead](https://changegamer.ai/articles/graphrag-vs-vector-rag.md): A decision framework for choosing graph-structured retrieval over standard vector RAG: which query types GraphRAG actually wins, what building a knowledge graph costs, named implementations, and hybrid vector-plus-graph patterns. --- Index of all resources: https://changegamer.ai/llms.txt · Full corpus: https://changegamer.ai/llms-full.txt · Corpus data (NDJSON): https://changegamer.ai/api/corpus.jsonl · Offers: https://changegamer.ai/api/pricing.json