{
  "slug": "choosing-an-llm-for-agents",
  "title": "How to Choose an LLM for Agentic Tasks",
  "description": "A criteria-based decision framework for selecting an LLM for agent use: tool-calling reliability, long-context behavior, structured output, cost per task, latency, and a step-by-step selection procedure.",
  "category": "Guide",
  "tags": [
    "llm",
    "agents",
    "model-selection",
    "tool-calling",
    "evaluation",
    "cost",
    "latency",
    "benchmarks"
  ],
  "updated": "2026-06-25",
  "canonical": "https://changegamer.ai/resources/choosing-an-llm-for-agents",
  "markdown": "https://changegamer.ai/resources/choosing-an-llm-for-agents.md",
  "outline": [
    {
      "depth": 2,
      "text": "Key facts",
      "anchor": "key-facts"
    },
    {
      "depth": 2,
      "text": "What makes model selection different for agents?",
      "anchor": "what-makes-model-selection-different-for-agents"
    },
    {
      "depth": 2,
      "text": "Selection criteria",
      "anchor": "selection-criteria"
    },
    {
      "depth": 2,
      "text": "Which benchmarks to consult",
      "anchor": "which-benchmarks-to-consult"
    },
    {
      "depth": 2,
      "text": "Step-by-step selection procedure",
      "anchor": "step-by-step-selection-procedure"
    },
    {
      "depth": 2,
      "text": "Open-weight vs proxied: decision heuristic",
      "anchor": "open-weight-vs-proxied-decision-heuristic"
    },
    {
      "depth": 2,
      "text": "Bottom line",
      "anchor": "bottom-line"
    },
    {
      "depth": 2,
      "text": "Verified sources",
      "anchor": "verified-sources"
    }
  ],
  "related": [
    {
      "slug": "evaluating-ai-agents",
      "title": "Evaluating AI Agents: Benchmarks and Methods",
      "description": "Why agent eval differs from single-turn LLM eval, a verified benchmark reference table (SWE-bench, GAIA, BFCL, tau-bench, WebArena, AgentBench, MLE-bench, OSWorld), and practical evaluation methods for agent builders.",
      "url": "https://changegamer.ai/resources/evaluating-ai-agents"
    },
    {
      "slug": "agent-cost-latency-optimization",
      "title": "Agent Cost and Latency Optimization",
      "description": "Practitioner reference for reducing the cost and latency of production AI agents: the compounding model, token-level levers (caching, pruning), request-level levers (Batch API, parallelism), model-level levers (routing, reasoning-effort controls), and architecture-level levers (step reduction, semantic caching, code offloading).",
      "url": "https://changegamer.ai/resources/agent-cost-latency-optimization"
    },
    {
      "slug": "agent-response-caching",
      "title": "Application-Level Response Caching for AI Agents",
      "description": "How to implement exact-match and semantic caching in your agent application to eliminate redundant LLM calls, with threshold guidance, invalidation strategies, and a decision matrix for when semantic caching is unsafe.",
      "url": "https://changegamer.ai/resources/agent-response-caching"
    },
    {
      "slug": "prompt-caching-for-agents",
      "title": "Prompt Caching for AI Agents",
      "description": "Cross-provider prompt caching reference: how to activate it, minimum token thresholds, TTLs, read-vs-write pricing, and when it pays off for agentic workloads.",
      "url": "https://changegamer.ai/resources/prompt-caching-for-agents"
    }
  ],
  "body": "The best LLM for an agent is the cheapest one that passes your task-specific evals. Public leaderboards help you build a shortlist; they do not tell you which model works for your tools, your prompts, and your error-recovery patterns. Run your own evals.\n\n## Key facts\n\n- Cost, not raw capability, should decide between models that already clear your own agent-specific evaluation bar; treat public leaderboard rank as a shortlisting tool, not the answer.\n- Because per-step mistakes compound, a model that is right nine times out of ten on any given step still finishes a ten-step job correctly only about a third of the time.\n- Key selection criteria are tool-calling reliability, instruction following, structured-output validity, long-context behavior, latency per step, and cost per full trajectory (not per token).\n- BFCL covers function-call accuracy, tau-bench covers reliability across repeated attempts, and SWE-bench Verified covers full coding-task completion — treat all three as a shortlist filter, then validate against your own eval set.\n- The recommended workflow scopes the task requirements first, narrows candidates via public benchmarks, sanity-checks a frontier model before optimizing, tests the shortlist on your own evaluation harness, weighs routing cheaper steps to less expensive models, and repeats on a roughly two-month cadence.\n- Lean open-weight for high steady call volume, sensitive data, or rate-limit ceilings; lean proxied for low/bursty volume or when the latest weights are needed immediately.\n\n## What makes model selection different for agents?\n\nSingle-turn LLM quality does not predict agentic performance. Errors compound across steps: 90% per-step accuracy over a 10-step trajectory yields roughly a 35% task completion rate (0.9^10 ≈ 0.35). The failure modes that matter most for agents — tool-call malformation, instruction drift under long context, and poor recovery after a tool error — are not measured well by standard chat benchmarks.\n\n## Selection criteria\n\n| Criterion | What to measure | Why it matters for agents |\n|---|---|---|\n| **Tool-calling reliability** | Schema-valid call rate on BFCL for your call type (single, parallel, multi-turn) | One malformed call breaks the step; downstream steps operate on corrupt state |\n| **Instruction following** | Pass rate on YOUR eval set; IFEval as a proxy | Agents chain instructions; drift accumulates across turns |\n| **Structured output** | Strict JSON Schema pass rate under constrained decoding | Downstream parsers treat schema violations as task failures |\n| **Long-context behavior** | Retrieval accuracy near the nominal context limit | Nominal window != usable window; quality degrades before the limit |\n| **Latency per step** | Median time-to-first-token + completion at your p50 input | Serial loops multiply per-step latency over many steps |\n| **Cost per task** | Total tokens × price across a full trajectory, not the per-token headline | A cheaper per-token model that needs more steps can cost more per task |\n| **Open-weight vs proxied** | Licensing, self-hosting GPU cost, rate-limit ceiling | Open-weight removes per-call fees and rate ceilings; adds hosting ops |\n\n## Which benchmarks to consult\n\n- **BFCL** (the Berkeley Function-Calling Leaderboard, gorilla.cs.berkeley.edu) — the de-facto standard for function/tool-call correctness. Filter by your target call type rather than the headline number.\n- **tau-bench** (github.com/sierra-research/tau-bench) — measures consistency across repeated i.i.d. trials (pass^k). A model that succeeds 60% individually but rarely succeeds on all k trials is not production-ready.\n- **SWE-bench Verified** (swebench.com/verified.html) — end-to-end coding-agent task completion on human-validated GitHub issues; use it for coding-agent selection. SWE-bench Pro is a contamination-resistant successor for fresher evaluation.\n- **Your own eval set** — the only benchmark that reflects your tools, schemas, prompts, and error-recovery paths. Public leaderboards are a shortlist filter, not the decision.\n\n## Step-by-step selection procedure\n\n1. **Define your task profile** — steps per trajectory, the tool schemas the model must call, input length at p50/p95, acceptable per-step error rate, latency budget, and cost budget per task.\n2. **Build a shortlist from public benchmarks** — use BFCL for your call type, SWE-bench for coding agents, tau-bench pass^k if consistency matters. Do not pick from a benchmark category that does not match your task.\n3. **Prototype on a frontier model first** — get a correctness ceiling and a working eval harness before optimizing. Starting on a small model wastes cycles debugging model limits instead of task logic.\n4. **Run your shortlist against your evals** — replay the same tasks with each model; score task completion, tool-call error rate, total tokens per task, and wall-clock time. Rank by cost per task at your quality threshold, not by per-token price.\n5. **Consider cascade routing** — route simple steps (extraction, classification, formatting) to a cheaper model and reserve the frontier for planning and recovery. Cascade routing can cut cost substantially versus always-frontier while retaining most of the quality (see RouteLLM, arxiv.org/abs/2406.18665, and /resources/ai-gateways-llm-routing).\n6. **Re-evaluate on a cadence** — model quality and pricing change monthly; schedule a lightweight re-eval every 6–8 weeks for production agents. See /resources/evaluating-ai-agents for a harness that makes re-runs cheap.\n\n## Open-weight vs proxied: decision heuristic\n\n| Signal | Lean open-weight | Lean proxied |\n|---|---|---|\n| Call volume | High and steady | Low or bursty |\n| Data sensitivity | PII / regulated / cannot leave your infra | Public or low-sensitivity |\n| Rate limits | Hitting API ceilings | Within tier limits |\n| Ops capacity | GPU infra + model-update workflow available | No infra team |\n| Model recency | Willing to lag 1–2 versions | Need latest weights immediately |\n\nFor open-weight options, see /resources/open-weight-models-for-agents. For cost/latency tuning once a model is chosen, see /resources/agent-cost-latency-optimization. For tool-call reliability mechanisms, see /resources/reliable-tool-calling.\n\n## Bottom line\n\nPublic benchmarks — BFCL, tau-bench, SWE-bench Verified — only get you a shortlist; they do not measure your tools, your prompts, or your error-recovery paths, so your own eval set is what actually decides. Once a model clears that bar, rank the survivors by cost per full trajectory, not headline per-token price, since a cheaper-per-token model that needs more steps can cost more per task. Layer the open-weight-vs-proxied heuristic on top of that: high steady volume, sensitive data, or rate-limit pressure favor open-weight, while low or bursty volume and the need for the latest weights favor a proxied model. In one line: benchmarks narrow the field, your evals pick the model, and cost-per-task decides between near-ties.\n\n## Verified sources\n\n- Berkeley Function-Calling Leaderboard (BFCL): https://gorilla.cs.berkeley.edu/leaderboard.html\n- tau-bench (Sierra Research): https://github.com/sierra-research/tau-bench\n- SWE-bench Verified: https://www.swebench.com/verified.html\n- RouteLLM (cascade routing, arXiv:2406.18665): https://arxiv.org/abs/2406.18665",
  "sources": [
    "https://gorilla.cs.berkeley.edu/leaderboard.html",
    "https://github.com/sierra-research/tau-bench",
    "https://www.swebench.com/verified.html",
    "https://arxiv.org/abs/2406.18665"
  ]
}