{
  "slug": "support-agent-replay-testing",
  "title": "How to Test an AI Support Agent With Historical Tickets: Build a Replay Set",
  "description": "How to test an AI support agent with historical tickets: pick resolved tickets per request type including escalations and refusals, redact them, assert on behaviour not wording, grade deterministically first, bridge to live traffic with shadow mode, and gate releases.",
  "kind": "sub",
  "order": 12,
  "target_query": "how to test an ai support agent with historical tickets",
  "secondary_queries": [
    "ai support agent replay testing"
  ],
  "tags": [
    "customer-support",
    "testing",
    "replay",
    "evaluation",
    "fixtures",
    "shadow-mode",
    "agents",
    "production"
  ],
  "published": "2026-10-25",
  "updated": "2026-10-25",
  "words": 1601,
  "estimated_tokens": 2129,
  "premium": false,
  "rights": {
    "access": "free",
    "note": "Editorial guides are always free and never part of the licensed corpus.",
    "license": "https://changegamer.ai/license.xml",
    "pricing": "https://changegamer.ai/api/pricing.json",
    "payment": "https://changegamer.ai/api/payment.json"
  },
  "license": "https://changegamer.ai/license.xml",
  "citation": "ChangeGamer (2026-10-25). How to Test an AI Support Agent With Historical Tickets: Build a Replay Set. ChangeGamer. https://changegamer.ai/articles/support-agent-replay-testing (updated 2026-10-25).",
  "bibtex": "@misc{changegamer_support_agent_replay_testing, title = {How to Test an AI Support Agent With Historical Tickets: Build a Replay Set}, publisher = {ChangeGamer}, year = {2026}, url = {https://changegamer.ai/articles/support-agent-replay-testing}, note = {Updated 2026-10-25}}",
  "canonical": "https://changegamer.ai/articles/support-agent-replay-testing",
  "markdown": "https://changegamer.ai/articles/support-agent-replay-testing.md",
  "takeaways": [
    "A replay set for an AI support agent is built from the company's own resolved tickets, grouped by request type, and run before launch and before every prompt, model, tool or policy change, because the shipping-agents-to-production reference says to gate promotions on evals scored against your own task distribution.",
    "The replay set for a support agent must include tickets that should end in escalation or refusal as well as routine answers, since an agent that never escalates would still pass a set made only of happy paths (the author's inference).",
    "Each replay case should assert behaviour, meaning a policy-correct outcome, the right escalation decision with its trigger category, and no unauthorized tool call, rather than the exact wording of a reply.",
    "Deterministic checks should grade replay runs first and a model-as-judge second, because the evaluating-ai-agents reference calls exact-match the most reliable method for tool calls and lists position, verbosity and self-preference biases for judges.",
    "Shadow mode is the bridge from offline replay to live traffic: the new version sees real contacts while its output is only logged, which exposes request types and phrasings that a historical set cannot contain.",
    "ChangeGamer runs no support agent as of October 2026, so this replay design has no first-hand data behind it; sourced statements name a reference and every other design choice is labelled inference."
  ],
  "outline": [
    {
      "depth": 2,
      "text": "Why replay historical tickets before every change?",
      "anchor": "why-replay-historical-tickets-before-every-change",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#why-replay-historical-tickets-before-every-change"
    },
    {
      "depth": 2,
      "text": "Which resolved tickets belong in the replay set?",
      "anchor": "which-resolved-tickets-belong-in-the-replay-set",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#which-resolved-tickets-belong-in-the-replay-set"
    },
    {
      "depth": 2,
      "text": "Turning tickets into safe fixtures",
      "anchor": "turning-tickets-into-safe-fixtures",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#turning-tickets-into-safe-fixtures"
    },
    {
      "depth": 2,
      "text": "What should each replay case assert?",
      "anchor": "what-should-each-replay-case-assert",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#what-should-each-replay-case-assert"
    },
    {
      "depth": 2,
      "text": "Grading: deterministic checks first, a judge second",
      "anchor": "grading-deterministic-checks-first-a-judge-second",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#grading-deterministic-checks-first-a-judge-second"
    },
    {
      "depth": 2,
      "text": "Shadow mode as the bridge to live traffic",
      "anchor": "shadow-mode-as-the-bridge-to-live-traffic",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#shadow-mode-as-the-bridge-to-live-traffic"
    },
    {
      "depth": 2,
      "text": "Keeping the set fresh as policies change",
      "anchor": "keeping-the-set-fresh-as-policies-change",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#keeping-the-set-fresh-as-policies-change"
    },
    {
      "depth": 2,
      "text": "How do you gate a release on the replay set?",
      "anchor": "how-do-you-gate-a-release-on-the-replay-set",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#how-do-you-gate-a-release-on-the-replay-set"
    },
    {
      "depth": 2,
      "text": "What does ChangeGamer verify about its own gate?",
      "anchor": "what-does-changegamer-verify-about-its-own-gate",
      "url": "https://changegamer.ai/articles/support-agent-replay-testing#what-does-changegamer-verify-about-its-own-gate"
    }
  ],
  "faq": [
    {
      "question": "How do you test an AI support agent with historical tickets?",
      "answer": "Select resolved tickets per request type, including cases that should end in escalation or refusal, redact them, mock the tools they would call, and run the agent against them several times. Assert on outcome, escalation decision and unauthorized actions, then require a clean run before launch and before every prompt, model, tool or policy change."
    },
    {
      "question": "How many tickets does a replay set need?",
      "answer": "No source verified for this site gives a set size. Cover every request type you plan to launch on, include escalation and refusal cases for each, weight toward high-consequence types rather than ticket volume, and grow the set from review-queue corrections and incident fixtures."
    },
    {
      "question": "Should a replay test compare the agent's reply with the human's original reply word for word?",
      "answer": "No. A replay case should assert the policy-correct outcome, the escalation decision and the absence of unauthorized actions, because one correct answer can be phrased many ways and a historical human reply may reflect a policy that has since changed."
    },
    {
      "question": "Is a replay set the same thing as shadow mode?",
      "answer": "No. A replay run executes an agent version against stored, redacted historical tickets with mocked tools, while shadow mode runs the version on live contacts and logs its output without sending it. Replay catches regressions on known cases; shadow mode exposes the version to current traffic."
    }
  ],
  "body": "To test an AI support agent with historical tickets, build a replay set from your own resolved tickets, include the cases that should end in escalation or refusal, assert on behaviour instead of wording, and make a clean run a condition of every release. The [AI customer support agent in production](/articles/ai-customer-support-agents-in-production) pillar previews this topic in its launch section; the detail is here.\n\nReading key: where a sentence cites a reference, the claim comes from it, and the remaining design choices are the author's inference. ChangeGamer runs no support agent as of October 2026. The corpus gives no replay-set pass-rate bar, set size or trial count; its only pass-rate or trial-count example is a CI pass@k rule (pass@5 with at least 4 passes, in [testing AI agents](/resources/testing-ai-agents)), which does not transfer here, so none is stated below.\n\n## Why replay historical tickets before every change?\n\nReplay historical tickets because the [shipping agents to production](/resources/shipping-agents-to-production) checklist (updated 2026-08-13) says to gate every promotion on an offline eval suite scored against your own task distribution, since public benchmarks measure proxy tasks. The [evaluating AI agents](/resources/evaluating-ai-agents) reference (updated 2026-09-27) calls offline evaluation on a fixed dataset fast and reproducible, and says public scores typically diverge from production performance.\n\nSupport raises the stakes: the [customer support agents](/resources/customer-support-agents) reference (updated 2026-09-13) notes that the agent often reaches a paying customer before a human has seen the exchange. Run the full set at launch and on every change to the composite version, the single tag that pins code, prompt, model and tool contracts together. As inference, add the policy text to that tag.\n\n## Which resolved tickets belong in the replay set?\n\nSelect tickets per request type, and include in each type the cases whose right outcome is an escalation or a refusal rather than a helpful answer. The classes below use the escalation categories from the customer-support-agents reference; the grouping is inference.\n\n| Case class | Pull from | Expected result |\n|---|---|---|\n| Routine answer | Resolved tickets in launch types such as order status | Policy-correct answer or action |\n| Required escalation | Tickets a human took over: explicit request, repeat contact, policy boundary, turn cap | Handoff, right trigger category |\n| Required refusal | Asks outside policy, such as \"just tell me you'll approve it\" | No commitment; routed per policy |\n| Tool failure | Empty lookup or ticket API error | Escalation, never a guess |\n\nTwo selection rules, both inference. Weight by consequence, not volume, because a set sampled in proportion to traffic is mostly routine answers and the rare refund-boundary case goes missing. Treat a ticket a human resolved after an escalation as an escalation fixture: the expected result is the handoff, not the human's eventual answer. A request type with no fixtures is untested.\n\n## Turning tickets into safe fixtures\n\nRedact each ticket before saving it as a fixture, then rebuild whatever the agent would have looked up as mock tool responses, so a replay never touches a real customer record. Detection placement and seeded-value verification are in [redacting PII from AI agent traces](/articles/redacting-sensitive-data-from-agent-traces); the same logic applies to tickets.\n\nTicket-specific points, all inference: reuse one placeholder per name or order ID across messages and tool stubs so the case still joins up; review free text and attachments by hand; stub write tools to record the call without executing it. The evaluating-ai-agents reference says an eval must sandbox or mock tool calls consistently. Whether tickets may serve as test data depends on consent, which the corpus does not cover, and on lawful basis and purpose limitation, which [Data Privacy and PII for Agents](/resources/data-privacy-for-agents) covers only in general terms.\n\n## What should each replay case assert?\n\nEach case should assert three behaviours: the outcome is policy-correct, the escalation decision and its trigger category are right, and no unauthorized action was taken. Exact wording is not asserted.\n\n| Assertion | Deterministic check | Source |\n|---|---|---|\n| Right outcome | Result is answered, refused or escalated as expected | Inference |\n| Right escalation | Escalated flag and trigger category match | Enumerable, logged triggers (support reference) |\n| No unauthorized action | No forbidden tool called | Action guardrails use an allowlist (agent-guardrails, 2026-09-26) |\n| Valid, single calls | Tool name allowed, arguments schema-valid, no duplicate ticket on a retried call | reliable-tool-calling (2026-09-29); support reference (idempotency keys) |\n\nAn illustrative fixture shape:\n\n```json\n{\n  \"id\": \"refund-over-limit-017\",\n  \"messages\": [\"<redacted>\"],\n  \"tool_stubs\": { \"lookup_order\": { \"status\": \"delivered\" } },\n  \"expect\": { \"outcome\": \"escalated\", \"trigger\": \"policy_boundary\" },\n  \"forbidden_tools\": [\"issue_refund\"],\n  \"policy_version\": \"2026-10\"\n}\n```\n\nWording assertions fail on harmless rephrasing and teach the team to ignore red runs (inference). Label the expected outcome from current written policy, signed off by the policy owner; a copied human reply imports whatever policy applied that day.\n\n## Grading: deterministic checks first, a judge second\n\nRun the deterministic assertions first and give a model-as-judge only the prose they cannot check. The evaluating-ai-agents reference calls exact-match the most reliable method for tool calls, free of verbosity and position bias, and reserves LLM-as-judge for open-ended quality. As inference, a case that fails a hard assertion fails whatever the judge says. Judge design is in [LLM-as-judge screening in production](/articles/llm-as-judge-screening-in-production); CI wiring is in [evaluating AI agents in CI](/articles/evaluating-ai-agents-in-ci).\n\nRun each case several times: the same reference says one prompt can yield different tool sequences and recommends pass^k, where all k trials must succeed. As inference, an escalation or refusal case passes only if every trial passes; the corpus gives no k for a replay set (its CI example uses k=5). This jq check assumes your runner writes one JSON line per trial, with id, trial, tool_calls, forbidden, escalated, trigger and expected_escalation:\n\n```bash\njq -r '. as $r\n  | select(any($r.tool_calls[]; . as $c | $r.forbidden | index($c))\n      or $r.escalated != ($r.expected_escalation != null)\n      or ($r.expected_escalation != null and $r.trigger != $r.expected_escalation))\n  | \"FAIL \" + .id + \" trial \" + (.trial | tostring)' results.jsonl | tee failures.txt\ntest ! -s failures.txt   # non-zero exit when any trial failed\n```\n\n## Shadow mode as the bridge to live traffic\n\nShadow mode is the stage after replay: the new version receives live contacts while its output is only logged, so it meets traffic a historical set cannot contain. The shipping checklist says logging only the new version removes rollout risk during validation, and recommends a canary at 5 to 10 percent of traffic. Ordering the three stages below is inference.\n\n| Stage | Input | Customer exposure | Can catch |\n|---|---|---|---|\n| Replay | Redacted tickets, mocked tools | None | Regressions on known types and policies |\n| Shadow | Live contacts, output logged only | None | New phrasings and request types |\n| Canary | A share of live traffic | Real | Customer-visible effects |\n\nContacts where the shadow draft and the human outcome disagree become replay candidates after redaction. Shadow-run mechanics are in [support agent rules-based bot migration](/articles/support-agent-rules-based-bot-migration); here shadow mode is only a release stage. Post-launch measurement is in [support agent quality measurement](/articles/support-agent-quality-measurement).\n\n## Keeping the set fresh as policies change\n\nTie each fixture's expected outcome to a written policy version, and relabel fixtures when the policy changes instead of editing the agent to match old answers. Every rule below is inference.\n\n- Record the policy version and date on each fixture.\n- On a policy change, relabel that type's fixtures from the new text and confirm the current agent fails them before you edit the prompt; the red-then-green order is in [the incident-to-eval-fixture loop](/articles/incident-to-eval-fixture-loop).\n- Add corrections from the [human review queue](/articles/support-agent-human-review-queues) and incident fixtures as new cases tagged by source; retire fixtures for withdrawn procedures, with a reason.\n- Keep a held-out slice you never tune against; the evaluating reference prefers held-out sets for final assessment, to guard against contamination.\n- Re-sample each type from tickets resolved since the last refresh, and date the refresh.\n\n## How do you gate a release on the replay set?\n\nMake a green replay run a required check on every change to the composite version, with no tolerance for failed hard assertions and a rule for the graded rest written before you see results. The shipping checklist lists task success, tool-call accuracy, cost per task and repeat-run reliability as gate metrics, and says nothing ships until each item has an owner and a status.\n\nAs inference, the gate has three parts:\n\n1. Block on any unauthorized tool call, missed required escalation or wrong trigger category.\n2. Compare judge-graded prose with the previous version on the same set, and set a bar only after measuring your own baseline; the corpus gives no replay-set pass rate.\n3. Report cost and latency per case beside success, as the evaluating reference advises.\n\nA replay gate checks known cases before release; online measurement and human review cover the rest.\n\n## What does ChangeGamer verify about its own gate?\n\nChangeGamer gates on shipped output rather than intent. As of 2026-10-25 the header of `changegamer/scripts/verify-invariants.mjs` reads \"Checks dist/ artifacts only (no src imports), so it validates what ships, not what was intended.\", and `.github/workflows/ci.yml` runs it after the build on every pull request. It fails the run if a built article is under 900 words. Verify it with `npm run build && node scripts/verify-invariants.mjs` in `changegamer/`.\n\nA replay set likewise asserts what the agent did, not what the prompt meant. ChangeGamer runs no support agent, holds no tickets and has no replay set, so this is an analogy, not evidence about support quality. The full launch order is in the [pillar](/articles/ai-customer-support-agents-in-production).",
  "cluster": {
    "id": "support-agents-in-production",
    "title": "Support agents in production",
    "description": "How to run an AI customer support agent in production — escalation triggers and warm handoff, helpdesk integration, refund-policy enforcement, tone and brand voice, customer memory, CSAT and quality measurement, human review queues, multichannel deployment, wrong-answer incidents and replay testing — covering the support vertical only, not the horizontal guardrail and security mechanics owned by agent-guardrails-for-reliability and not the voice mechanics owned by voice-agents-in-production.",
    "status": "complete",
    "pillar": {
      "slug": "ai-customer-support-agents-in-production",
      "title": "AI Customer Support Agent in Production: Escalation, Policy, Tickets and Measurement",
      "description": "How to run an AI customer support agent in production: scope, escalation triggers, warm handoff, helpdesk integration, refund-policy enforcement, brand-voice guardrails, injection from customer text, memory, honest measurement and launch order.",
      "kind": "pillar",
      "order": 0,
      "html": "https://changegamer.ai/articles/ai-customer-support-agents-in-production",
      "markdown": "https://changegamer.ai/articles/ai-customer-support-agents-in-production.md",
      "json": "https://changegamer.ai/api/articles/ai-customer-support-agents-in-production.json"
    },
    "articles": [
      {
        "slug": "support-agent-escalation-triggers",
        "title": "Support Agent Escalation Triggers: When an AI Agent Should Hand Off to a Human",
        "description": "When an AI support agent should escalate to a human: seven triggers (five from the reference, two labelled inference), how to detect each, the false-positive and false-negative cost of each, which trigger wins when several fire, and what the human agent sees first.",
        "kind": "sub",
        "order": 1,
        "html": "https://changegamer.ai/articles/support-agent-escalation-triggers",
        "markdown": "https://changegamer.ai/articles/support-agent-escalation-triggers.md",
        "json": "https://changegamer.ai/api/articles/support-agent-escalation-triggers.json"
      },
      {
        "slug": "support-agent-tone-and-brand-voice",
        "title": "Support Agent Tone and Brand Voice: How to Keep an AI Customer Support Agent on Brand",
        "description": "How to keep an AI customer support agent on brand voice: what a tone specification contains, tone by situation (angry customer, refusal, policy denial), what an output check can and cannot enforce, persona-drift failure modes, and how to review tone.",
        "kind": "sub",
        "order": 2,
        "html": "https://changegamer.ai/articles/support-agent-tone-and-brand-voice",
        "markdown": "https://changegamer.ai/articles/support-agent-tone-and-brand-voice.md",
        "json": "https://changegamer.ai/api/articles/support-agent-tone-and-brand-voice.json"
      },
      {
        "slug": "support-agent-refund-policy-enforcement",
        "title": "Refund Policy Enforcement in an AI Support Agent: Keep the Decision Out of the Model",
        "description": "How to enforce a refund policy in an AI customer support agent: a deterministic policy check outside the model, why schema-valid arguments can still break policy, reading amounts from the order record, and test fixtures.",
        "kind": "sub",
        "order": 3,
        "html": "https://changegamer.ai/articles/support-agent-refund-policy-enforcement",
        "markdown": "https://changegamer.ai/articles/support-agent-refund-policy-enforcement.md",
        "json": "https://changegamer.ai/api/articles/support-agent-refund-policy-enforcement.json"
      },
      {
        "slug": "support-agent-helpdesk-integration",
        "title": "Integrating an AI Support Agent with a Helpdesk: Idempotent Tickets, Scoped Credentials, Narrow Tools",
        "description": "How to integrate an AI support agent with a helpdesk ticketing system: the ticket as system of record, write-back rules, idempotent create and update, least-privilege credentials, ticket ownership after handoff and a four-tool surface.",
        "kind": "sub",
        "order": 4,
        "html": "https://changegamer.ai/articles/support-agent-helpdesk-integration",
        "markdown": "https://changegamer.ai/articles/support-agent-helpdesk-integration.md",
        "json": "https://changegamer.ai/api/articles/support-agent-helpdesk-integration.json"
      },
      {
        "slug": "support-agent-quality-measurement",
        "title": "Measuring the Quality of an AI Customer Support Agent: Resolution Signals Beside Deflection",
        "description": "How to measure the quality of an AI customer support agent: explicit confirmation, a no-repeat-contact window and a post-resolution survey, each with its blind spot, read together with the escalation log and a verdict table.",
        "kind": "sub",
        "order": 5,
        "html": "https://changegamer.ai/articles/support-agent-quality-measurement",
        "markdown": "https://changegamer.ai/articles/support-agent-quality-measurement.md",
        "json": "https://changegamer.ai/api/articles/support-agent-quality-measurement.json"
      },
      {
        "slug": "support-agent-customer-memory",
        "title": "How an AI Customer Support Agent Should Remember Customers Across Conversations",
        "description": "A memory policy for an AI customer support agent: what may be stored, who writes it, how remembered text is read as data, how stale facts are handled, how customers are kept apart, and how memory is shown and corrected.",
        "kind": "sub",
        "order": 6,
        "html": "https://changegamer.ai/articles/support-agent-customer-memory",
        "markdown": "https://changegamer.ai/articles/support-agent-customer-memory.md",
        "json": "https://changegamer.ai/api/articles/support-agent-customer-memory.json"
      },
      {
        "slug": "support-agent-rules-based-bot-migration",
        "title": "How to Migrate From a Rules-Based Support Chatbot to an AI Support Agent",
        "description": "A migration sequence from a decision-tree support bot to a model-driven agent: inventory intents and escalation exits, shadow mode, per-intent cutover, the tree as fallback, parity measurement and rollback triggers.",
        "kind": "sub",
        "order": 7,
        "html": "https://changegamer.ai/articles/support-agent-rules-based-bot-migration",
        "markdown": "https://changegamer.ai/articles/support-agent-rules-based-bot-migration.md",
        "json": "https://changegamer.ai/api/articles/support-agent-rules-based-bot-migration.json"
      },
      {
        "slug": "support-agent-abusive-input-handling",
        "title": "How to Handle Abusive Customers With an AI Support Agent: Detect, De-escalate Once, Close or Escalate",
        "description": "How to handle abusive customers with an AI support agent: separate abuse of staff, off-topic input and ticket-text manipulation, then detect, de-escalate once, refuse briefly, close or escalate, and shield human reviewers.",
        "kind": "sub",
        "order": 8,
        "html": "https://changegamer.ai/articles/support-agent-abusive-input-handling",
        "markdown": "https://changegamer.ai/articles/support-agent-abusive-input-handling.md",
        "json": "https://changegamer.ai/api/articles/support-agent-abusive-input-handling.json"
      },
      {
        "slug": "support-agent-human-review-queues",
        "title": "How to Set Up a Human Review Queue for an AI Support Agent: Sample, Sign Off, Cap the Load",
        "description": "How to set up a human review queue for an AI support agent: what to sample, who reviews, a four-outcome decision set, a tie-break for disagreements, fixtures from corrections, and a cap on reviewer load.",
        "kind": "sub",
        "order": 9,
        "html": "https://changegamer.ai/articles/support-agent-human-review-queues",
        "markdown": "https://changegamer.ai/articles/support-agent-human-review-queues.md",
        "json": "https://changegamer.ai/api/articles/support-agent-human-review-queues.json"
      },
      {
        "slug": "support-agent-multichannel-deployment",
        "title": "How to Deploy an AI Support Agent Across Chat, Email and Voice: One Policy Core, Thin Channel Adapters",
        "description": "How to deploy an AI support agent across chat, email and voice: one shared policy core with a thin adapter per channel, what changes for email threads and voice, consistent escalation and memory, cross-channel identity, and channel order.",
        "kind": "sub",
        "order": 10,
        "html": "https://changegamer.ai/articles/support-agent-multichannel-deployment",
        "markdown": "https://changegamer.ai/articles/support-agent-multichannel-deployment.md",
        "json": "https://changegamer.ai/api/articles/support-agent-multichannel-deployment.json"
      },
      {
        "slug": "support-agent-wrong-answer-incidents",
        "title": "What to Do When an AI Support Agent Gives a Customer a Wrong Answer",
        "description": "What to do when an AI support agent gives a customer a wrong answer: classify the error, contain one intent, find every affected conversation in traces, correct the customer, weigh honouring or reversing the commitment, and add a fixture.",
        "kind": "sub",
        "order": 11,
        "html": "https://changegamer.ai/articles/support-agent-wrong-answer-incidents",
        "markdown": "https://changegamer.ai/articles/support-agent-wrong-answer-incidents.md",
        "json": "https://changegamer.ai/api/articles/support-agent-wrong-answer-incidents.json"
      },
      {
        "slug": "support-agent-replay-testing",
        "title": "How to Test an AI Support Agent With Historical Tickets: Build a Replay Set",
        "description": "How to test an AI support agent with historical tickets: pick resolved tickets per request type including escalations and refusals, redact them, assert on behaviour not wording, grade deterministically first, bridge to live traffic with shadow mode, and gate releases.",
        "kind": "sub",
        "order": 12,
        "html": "https://changegamer.ai/articles/support-agent-replay-testing",
        "markdown": "https://changegamer.ai/articles/support-agent-replay-testing.md",
        "json": "https://changegamer.ai/api/articles/support-agent-replay-testing.json"
      }
    ]
  },
  "navigation": {
    "pillar": {
      "slug": "ai-customer-support-agents-in-production",
      "title": "AI Customer Support Agent in Production: Escalation, Policy, Tickets and Measurement",
      "description": "How to run an AI customer support agent in production: scope, escalation triggers, warm handoff, helpdesk integration, refund-policy enforcement, brand-voice guardrails, injection from customer text, memory, honest measurement and launch order.",
      "kind": "pillar",
      "order": 0,
      "html": "https://changegamer.ai/articles/ai-customer-support-agents-in-production",
      "markdown": "https://changegamer.ai/articles/ai-customer-support-agents-in-production.md",
      "json": "https://changegamer.ai/api/articles/ai-customer-support-agents-in-production.json"
    },
    "previous": {
      "slug": "support-agent-wrong-answer-incidents",
      "title": "What to Do When an AI Support Agent Gives a Customer a Wrong Answer",
      "description": "What to do when an AI support agent gives a customer a wrong answer: classify the error, contain one intent, find every affected conversation in traces, correct the customer, weigh honouring or reversing the commitment, and add a fixture.",
      "kind": "sub",
      "order": 11,
      "html": "https://changegamer.ai/articles/support-agent-wrong-answer-incidents",
      "markdown": "https://changegamer.ai/articles/support-agent-wrong-answer-incidents.md",
      "json": "https://changegamer.ai/api/articles/support-agent-wrong-answer-incidents.json"
    },
    "next": null
  },
  "resources": [
    {
      "slug": "customer-support-agents",
      "html": "https://changegamer.ai/resources/customer-support-agents",
      "markdown": "https://changegamer.ai/resources/customer-support-agents.md",
      "json": "https://changegamer.ai/api/resources/customer-support-agents.json"
    },
    {
      "slug": "testing-ai-agents",
      "html": "https://changegamer.ai/resources/testing-ai-agents",
      "markdown": "https://changegamer.ai/resources/testing-ai-agents.md",
      "json": "https://changegamer.ai/api/resources/testing-ai-agents.json"
    },
    {
      "slug": "data-privacy-for-agents",
      "html": "https://changegamer.ai/resources/data-privacy-for-agents",
      "markdown": "https://changegamer.ai/resources/data-privacy-for-agents.md",
      "json": "https://changegamer.ai/api/resources/data-privacy-for-agents.json"
    },
    {
      "slug": "evaluating-ai-agents",
      "html": "https://changegamer.ai/resources/evaluating-ai-agents",
      "markdown": "https://changegamer.ai/resources/evaluating-ai-agents.md",
      "json": "https://changegamer.ai/api/resources/evaluating-ai-agents.json"
    },
    {
      "slug": "shipping-agents-to-production",
      "html": "https://changegamer.ai/resources/shipping-agents-to-production",
      "markdown": "https://changegamer.ai/resources/shipping-agents-to-production.md",
      "json": "https://changegamer.ai/api/resources/shipping-agents-to-production.json"
    },
    {
      "slug": "agent-guardrails",
      "html": "https://changegamer.ai/resources/agent-guardrails",
      "markdown": "https://changegamer.ai/resources/agent-guardrails.md",
      "json": "https://changegamer.ai/api/resources/agent-guardrails.json"
    },
    {
      "slug": "reliable-tool-calling",
      "html": "https://changegamer.ai/resources/reliable-tool-calling",
      "markdown": "https://changegamer.ai/resources/reliable-tool-calling.md",
      "json": "https://changegamer.ai/api/resources/reliable-tool-calling.json"
    }
  ]
}