#evaluation

7 resources tagged #evaluation on ChangeGamer.

Evaluating AI Agents: Benchmarks and Methods · Reference
Why agent eval differs from single-turn LLM eval, a verified benchmark reference table (SWE-bench, GAIA, BFCL, tau-bench, WebArena, AgentBench, MLE-bench, OSWorld), and practical evaluation methods for agent builders.
Agent Observability and Tracing · Guide
Why agents need observability beyond app logs, how OpenTelemetry GenAI semantic conventions model agent runs as traces, key signals to capture, and a verified tooling landscape.
RAG and Retrieval for Agents · Guide
End-to-end practitioner reference for Retrieval-Augmented Generation: pipeline stages, chunking strategies, dense/sparse/hybrid retrieval, reranking, agentic retrieval patterns, quality failure modes, and evaluation — with verified sources for every named technique.
Shipping AI Agents to Production: A Production-Readiness Checklist · Guide
End-to-end checklist for productionizing an AI agent — evaluation gates, observability, guardrails, cost controls, resilience, durability, HITL approvals, secrets, rollback, and incident response.
Reranking for RAG: Cross-Encoders, LLM Rerankers, and Hosted APIs · Guide
Second-stage retrieval step that re-scores bi-encoder candidates with full query-document attention, boosting precision without sacrificing recall; covers cross-encoder, LLM, and late-interaction reranking, hosted APIs, sizing heuristics, and evaluation.
How to Choose an LLM for Agentic Tasks · Guide
A criteria-based decision framework for selecting an LLM for agent use: tool-calling reliability, long-context behavior, structured output, cost per task, latency, and a step-by-step selection procedure.
Prompt Management and Versioning: The Ops of Prompts in Production · Guide
Treating prompts as deployable artifacts: versioning, external registries, A/B and canary testing, eval-gated promotion, rollback, and the composite-version problem.