#cost
5 resources and 7 guides tagged #cost on ChangeGamer. Includes Agent Cost and Latency Optimization and Prompt Caching for AI Agents.
- Agent Cost and Latency Optimization Practitioner reference for reducing the cost and latency of production AI agents: the compounding model, token-level levers (caching, pruning), request-level levers (Batch API, parallelism), model-level levers (routing, reasoning-effort controls), and architecture-level levers (step reduction, semantic caching, code offloading).
- Prompt Caching for AI Agents Cross-provider prompt caching reference: how to activate it, minimum token thresholds, TTLs, read-vs-write pricing, and when it pays off for agentic workloads.
- How to Choose an LLM for Agentic Tasks A criteria-based decision framework for selecting an LLM for agent use: tool-calling reliability, long-context behavior, structured output, cost per task, latency, and a step-by-step selection procedure.
- Application-Level Response Caching for AI Agents How to implement exact-match and semantic caching in your agent application to eliminate redundant LLM calls, with threshold guidance, invalidation strategies, and a decision matrix for when semantic caching is unsafe.
- LLM Tokenization and Token Counting: tiktoken, Provider Counting Endpoints, and Why Counts Differ Decision rule for counting LLM tokens: bill from the response usage fields, gate requests with the provider's token-counting endpoint, estimate offline with a tokenizer library such as tiktoken, and never reuse a count across models. Includes pitfalls and budgeting rules.
Guides
- MCP Server Cost Optimization: Toolset Size, Caching Hints, and Fan-Out How the token cost of an MCP server's tool list, the 2026-07-28 spec's ttlMs/cacheScope caching hints, fan-out from callers you do not control, and per-tool-name cost visibility each shape what a production MCP server actually costs to run.
- How to Cut RAG Cost and Latency Without Cutting Quality RAG cost and latency as engineered budgets: where the money actually goes, caching layers and their hit-rate economics, routing queries to right-sized models, bounding retrieval fan-out, the hidden lines (reindex migrations, eval compute), and p95 discipline.
- AI Agent Observability and the Production Evaluation Playbook AI agent observability and evaluation once an agent is already live: tracing spans, retrieval-attribution logs, judge-based screening vs. human acceptance, online metrics, cost telemetry, and incident postmortems that feed evaluation fixtures.
- How to Track AI Agent Costs in Production How to track AI agent costs in production: version your price table, separate cached from uncached tokens, keep failed runs in the books, attribute shared costs, reconcile against the invoice, and alert on rate of spend.
- How to Compute Your Voice Agent Cost per Minute How to compute a voice agent cost per minute from your own pilot: symbolic formulas for native speech-to-speech and cascaded STT, LLM and TTS, the variables that move the result, idle audio, retries, tool-call turns, and a sensitivity check.
- Document Extraction Cost per Page: Tokens, Retries, Review and Budget Alarms How to model document extraction cost per page beyond the parse fee: page-count mix versus per-page pricing, image tokens versus per-call overhead, retries by failure class, review share, batch versus sync, and a budget kill switch on cost per accepted document.
- Browser Agent Cost per Task: A Formula, the Levers and the Latency Caveat How to compute browser agent cost per task as of 14 November 2026: a formula for screenshots and toolset overhead, an illustrative worked example, Anthropic image-token sizes, the cost levers and why the vendor pages read on 14 November 2026 give no latency figure.