# How to Monitor a Voice Agent in Production: Dashboard, Alerts and Runbook

> How to monitor a voice agent in production: per-stage latency percentiles, transcription confidence, containment paired with a resolution signal, live false-interruption rate, a handoff trigger review, and call-audio retention, each with what to chart, alert on and do.

Guide: Voice agents in production — part 9
Published: 2026-10-12 · Updated: 2026-10-12 · 1305 words · ~1736 tokens (estimate)
Canonical: https://changegamer.ai/articles/voice-agent-online-monitoring
JSON: https://changegamer.ai/api/articles/voice-agent-online-monitoring.json
Pillar: https://changegamer.ai/articles/voice-ai-agent-in-production.md

## In short

- As the author's inference, a voice agent dashboard should chart latency percentiles per pipeline stage, not only end to end, because a drifting stage can hide inside a healthy-looking total until callers notice.
- The evaluating-voice-agents reference names transcription confidence, latency per stage and overall, and containment as the online signals, so those three are the minimum live dashboard for any voice agent.
- Containment alone can look healthy while callers give up, so a voice agent dashboard should pair it with a resolution or repeat-contact signal before anyone treats a rise as an improvement.
- Every voice agent alert should be tied to a named action, and as the author's inference thresholds should be set from the agent's own baseline, since the corpus gives no safe numeric alert levels.
- The corpus does not document voice-specific alerting, so the chart, alert and action pairings in this article are the author's inference and should be rehearsed with a deliberate fault before launch.

---

A voice agent needs a live dashboard, a short list of alerts and a runbook that says what to do when each one fires; as inference, the failures are audible to callers before a text error log shows anything. This article is the operator's version of the monitoring half of the [voice AI agent in production](/articles/voice-ai-agent-in-production) pillar. Metric definitions live in the [evaluating voice agents](/resources/evaluating-voice-agents) reference and are not restated here; the question here is what to put on a screen and what to do about it.

Labels: "corpus" is a claim from a named reference, "inference" is the author's reasoning. The corpus names the signals but gives no alert levels, so every threshold below is "set from your own baseline".

## What belongs on a voice agent dashboard?

A voice agent dashboard needs five panels: latency by stage, transcription confidence, containment next to a resolution signal, false interruptions, and handoff triggers. The first, second and third panels come from the online signals the evaluating-voice-agents reference lists (corpus); the panel layout is inference.

| Panel | Chart | Alert on | Action that follows |
|---|---|---|---|
| Stage latency | P50 and P95 per stage | Drift from baseline or budget | Find the stage, then the cause |
| Transcription confidence | Distribution over calls | Shift in the distribution | Sample low-confidence calls |
| Containment and resolution | Both on one axis | They diverge | Review sampled ended calls |
| False interruptions | Rate per day | A step change | Revisit turn detection |
| Handoff triggers | Count by category | A new or surging category | Read the logged calls |

Tie each alert to a named person and a first step, or delete it. An alert nobody can act on trains the team to ignore the next one.

## How do you alert on per-stage latency percentiles?

Alert when any stage drifts from its own baseline or when the total breaches the budget; the split and the timestamping method are in the [voice agent latency budget](/articles/voice-agent-latency-budget), and the per-stage alerting rule is inference.

- **Chart:** each stage and the total, using that article's percentiles.
- **Alert on:** a stage whose P95 moves away from its own recent baseline even if the total still passes, plus any sustained breach of the total.
- **Action:** open a sampled slow call, read its trace to find the stage, then check the usual suspects: a tool call that stalled, a prompt that grew, or a vendor change. The [agent-observability reference](/resources/agent-observability) describes the trace-and-span model and says to attach exceptions to the span that failed, which is how a slow stage becomes findable.

As inference, charting only the total can hide a drifting stage, since stage changes may offset each other. For generic drift mechanics see [online agent metrics and drift monitoring](/articles/online-agent-metrics-and-drift-monitoring).

## What do you do when transcription confidence drops?

Treat a shift in the transcription confidence distribution as an early warning that the audio, not the model, has changed, and respond by sampling the low-confidence calls. The reference lists transcription confidence as an online signal (corpus); what a drop means is inference.

- **Chart:** the distribution of per-call confidence, split by whatever you can observe cheaply, such as telephony route or region.
- **Alert on:** a change in the distribution's shape, not a single low call.
- **Action:** listen to a sample from the affected slice. A route-specific drop points at the audio path; a drop everywhere after a date points at a vendor or model change.

Transcription confidence is the score your speech-to-text provider exposes; the corpus does not define it, so compare it only within one vendor and model version (inference). A mis-heard value that reaches a tool call is more costly than a mis-heard filler word, so also chart how often critical values such as order numbers were confirmed back, a design covered in [voice agent guardrails for spoken output](/articles/guardrails-for-spoken-output-in-voice-agents).

## How do you read containment without fooling yourself?

Read containment only next to a resolution signal, because a contained call includes callers who quit. The customer-support reference, as quoted in the pillar, cautions that a deflection count cannot separate resolution from abandonment (corpus); the pairing is the remedy.

- **Chart:** containment and a repeat-contact rate for the same caller or topic on one shared time axis. If you cannot link repeat calls, chart a weekly sample of ended calls graded for whether the issue was resolved.
- **Alert on:** containment rising while repeat contact also rises, or while sampled resolution falls.
- **Action:** read the sampled calls that ended without a handoff. Check whether callers hung up mid-task, and whether a refusal or loop pushed them out.

The grade of a sampled call is a judgment, so write the rubric down once and reuse it, or the trend measures the grader. Judge-style rubrics are discussed in [agent observability and the production evaluation playbook](/articles/agent-observability-and-evaluation), but writing the rubric for your own sampled calls is a step that playbook does not spell out.

## How do you watch false interruptions on live calls?

Chart false-interruption rate daily and alert on a step change, because a sudden jump usually follows a configuration, vendor or audio-path change. The reference names false-interruption rate and barge-in accuracy among end-to-end measures (corpus); the live version is inference.

- **Chart:** the rate per day, plus count of agent turns cut off early in the utterance if your logs can show it (define the window yourself, inference).
- **Alert on:** a step change aligned with a deploy date, not slow noise.
- **Action:** replay affected calls through the test method in [how to test turn detection and barge-in](/articles/voice-turn-detection-and-barge-in-testing), then compare the failing audio with your offline set.

Live data carries no ground truth on what the caller meant, so a person has to label a sample before the number is trusted. A rising rate with no deploy suggests the callers or their devices changed; add those clips to the offline set.

## How do you review the handoff trigger log?

Review the handoff trigger log weekly by category, because a new or growing category reveals a gap in the agent faster than any aggregate. Log contents and transfer mechanics are in [voice agent human handoff](/articles/voice-agent-human-handoff); only the review rules follow.

1. **Chart** handoffs by trigger category.
2. **Alert on** a category appearing for the first time or taking a growing share, and on any handoff with a missing payload field.
3. **Act** by reading a handful of calls in that category and deciding whether to fix the agent, change the trigger, or accept it.

Treat a falling handoff count with suspicion until the resolution panel agrees. Fewer transfers can mean the agent is better, or that callers can no longer reach a person.

## How long should call audio be kept and how should it be redacted?

Keep call audio only for sampled or flagged calls, for a retention period you state in writing, with redaction applied before anyone reviews it. That is inference; the pillar says to decide retention before launch, and the redaction method is in [redacting sensitive data from agent traces](/articles/redacting-sensitive-data-from-agent-traces).

## What does ChangeGamer know first-hand about monitoring a voice agent?

Nothing about operating one: as of October 2026 ChangeGamer runs no voice agent and has no call data, so no figure above is measured. The one verifiable fact is about its own build: `node scripts/verify-invariants.mjs`, run in `changegamer/` after `npm run build`, prints "All invariants hold." when its checks pass, and an alert that can be checked as mechanically as that one is the model to copy: it passes or fails with no judgment call. For the full decision guide, return to the [voice agents cluster](/articles/voice-ai-agent-in-production), and for the realtime vendor landscape see the [voice and realtime agents](/resources/voice-realtime-agents) reference.

## Frequently asked questions

### What should you monitor on a voice agent in production?

Monitor latency percentiles per pipeline stage, transcription confidence, containment with a resolution signal, false-interruption rate and handoff triggers. The evaluating-voice-agents reference names the first three as online signals; the rest are the author's inference.

### What latency alert threshold should a voice agent use?

Alert on drift from your own measured baseline per stage, compared with the budget you set. The corpus cites a sub-600 ms end-to-end target but no per-stage alert level, so any fixed number in a vendor blog should be treated as a starting guess.

### Is containment rate a good success metric for a voice agent?

Not alone. The customer-support reference cited in the pillar cautions that a contained call includes callers who gave up, so pair containment with a repeat-contact or sampled resolution review before reading a rise as an improvement.

### Do you need to record call audio to monitor a voice agent?

As the author's inference, no: per-stage timings, transcripts with confidence and trigger logs support most monitoring without audio. Keep audio only for sampled or flagged calls, under a stated retention period and with redaction applied.


---

## The rest of this guide

- [Voice AI Agent in Production: Architecture, Latency, Handoff and Launch](https://changegamer.ai/articles/voice-ai-agent-in-production.md): How to build and run a voice AI agent in production: cascaded vs speech-to-speech architecture, turn detection and barge-in, the latency budget, tool calls in a voice loop, transport, cost, human handoff and launch sequencing.
- [Voice Agent Latency Budget: Where the 600 ms Goes and How to Hold It](https://changegamer.ai/articles/voice-agent-latency-budget.md): How to split a 600 ms voice agent latency budget across speech-to-text, the language model, speech synthesis and tool calls, how to measure each stage, and which levers hold the budget.
- [How to Test Voice Agent Turn Detection and Barge-In](https://changegamer.ai/articles/voice-turn-detection-and-barge-in-testing.md): How to test voice agent turn detection and barge-in: build a labelled clip set, count barge-in accuracy and false-interruption rate, separate cut-off from talk-over failures, and split offline gates from online monitors.
- [What to Log and Filter in a Voice Agent, by Architecture](https://changegamer.ai/articles/voice-agent-architecture-cascaded-vs-speech-to-speech.md): What to log and filter in a voice agent depending on architecture: where a cascaded pipeline gives you text seams for logging, redaction and output filters, what a native speech-to-speech model leaves you to do from outside, and what a model swap forces you to re-test.
- [Function Calling in a Voice Agent: What a Tool May Be in a Spoken Loop](https://changegamer.ai/articles/tools-in-a-voice-agent-loop.md): Function calling in a voice agent: which vendors and frameworks support tools as of the corpus's 2026-07-09 verification, how to classify tools as lookup, action or handoff, what barge-in does to a side effect, and a checklist for admitting a tool.
- [Voice Agent Transport: WebRTC vs WebSocket, and Where SIP Fits](https://changegamer.ai/articles/voice-agent-transport-webrtc-vs-websocket-vs-sip.md): WebRTC vs WebSocket for voice AI agents: which transport each vendor documents as of the corpus's 2026-07-09 verification, how cascaded stacks use WebSocket components, a decision tree by client type, and a per-transport test checklist.
- [Voice Agent Human Handoff: What Happens on the Call When the Agent Lets Go](https://changegamer.ai/articles/voice-agent-human-handoff.md): Voice agent human handoff mechanics: the handoff as a tool that ends the agent's session, barge-in during the transfer announcement, what the caller hears on hold and on failure, a context payload schema, and a real-call test checklist.
- [How to Compute Your Voice Agent Cost per Minute](https://changegamer.ai/articles/voice-agent-cost-per-minute.md): How to compute a voice agent cost per minute from your own pilot: symbolic formulas for native speech-to-speech and cascaded STT, LLM and TTS, the variables that move the result, idle audio, retries, tool-call turns, and a sensitivity check.
- [Voice Agent Guardrails: Checking What the Agent Says Before the Caller Hears It](https://changegamer.ai/articles/guardrails-for-spoken-output-in-voice-agents.md): Voice agent guardrails for spoken output: sentence-level gating before TTS, fail-closed judges, spoken numbers a text PII filter misses, readback before state changes, transcription-driven injection, refusal phrasing for the ear, and an adversarial test list.
- [How to Handle Voice AI Vendor Model Changes Without Regressing Callers](https://changegamer.ai/articles/voice-agent-vendor-change-management.md): How to handle voice AI vendor model changes: classify the change, record the model id on every trace, replay an offline set with hard calls on old and new, canary before cutover, keep a rollback, and run a deprecation-readiness routine.
- [Voice Agent Launch Checklist: The Go/No-Go Gates Before Real Callers](https://changegamer.ai/articles/voice-agent-launch-checklist.md): A voice agent launch checklist as yes/no gates: scope, offline replay, per-stage latency, barge-in, tools, transport, handoff, cost, guardrails, monitoring and rollback, each with a pass signal and the article that owns the method.
- [How Does a Voice Agent Handle Callers Who Speak Another Language?](https://changegamer.ai/articles/voice-agent-multilingual-callers.md): How a live voice agent handles callers who speak another language: detecting the language, choosing speech components per language, slicing evaluation by language and handing off when the language is unsupported, with the thin corpus evidence stated plainly.

## Reference resources

- https://changegamer.ai/resources/evaluating-voice-agents.md
- https://changegamer.ai/resources/agent-observability.md
- https://changegamer.ai/resources/voice-realtime-agents.md

All guides: https://changegamer.ai/api/articles.json · Reference corpus: https://changegamer.ai/llms.txt
Licensing: https://changegamer.ai/api/pricing.json (offer catalog) · https://changegamer.ai/api/payment.json (payment methods, HTTP 402 flow) · access guide: https://changegamer.ai/resources/access-and-pricing.md