#fixtures
4 guides tagged #fixtures on ChangeGamer. Includes How to Turn an AI Agent Incident into an Evaluation Test Case.
Guides
- How to Turn an AI Agent Incident into an Evaluation Test Case How to turn an AI agent incident into an evaluation test case: freeze the trace, redact it, minimize it to the failing decision, label the expected outcome with a trusted oracle, prove it fails then passes, and retire it later.
- How to Set Up a Human Review Queue for an AI Support Agent: Sample, Sign Off, Cap the Load How to set up a human review queue for an AI support agent: what to sample, who reviews, a four-outcome decision set, a tie-break for disagreements, fixtures from corrections, and a cap on reviewer load.
- What to Do When an AI Support Agent Gives a Customer a Wrong Answer What to do when an AI support agent gives a customer a wrong answer: classify the error, contain one intent, find every affected conversation in traces, correct the customer, weigh honouring or reversing the commitment, and add a fixture.
- How to Test an AI Support Agent With Historical Tickets: Build a Replay Set How to test an AI support agent with historical tickets: pick resolved tickets per request type including escalations and refusals, redact them, assert on behaviour not wording, grade deterministically first, bridge to live traffic with shadow mode, and gate releases.