# How to Test Voice Agent Turn Detection and Barge-In

> How to test voice agent turn detection and barge-in: build a labelled clip set, count barge-in accuracy and false-interruption rate, separate cut-off from talk-over failures, and split offline gates from online monitors.

Guide: Voice agents in production — part 2
Published: 2026-10-08 · Updated: 2026-10-08 · 1374 words · ~1827 tokens (estimate)
Canonical: https://changegamer.ai/articles/voice-turn-detection-and-barge-in-testing
JSON: https://changegamer.ai/api/articles/voice-turn-detection-and-barge-in-testing.json
Pillar: https://changegamer.ai/articles/voice-ai-agent-in-production.md

## In short

- Testing voice agent barge-in means replaying labelled audio clips, each marked as a genuine interruption, a backchannel or noise, and counting how often the agent responded correctly to each label.
- The evaluating-voice-agents reference names two end-to-end quantities for interruption handling, barge-in accuracy and false-interruption rate, but states no threshold and no computation recipe for either.
- A voice agent can fail in two opposite ways, cutting the caller off or talking over the caller, and a single averaged score hides which of the two a release has made worse.
- A cascaded voice pipeline has a testable seam where barge-in flushes the synthesized-audio buffer, while a native speech-to-speech model is most practically tested from outside as a black box, which is an inference rather than a corpus finding.
- Offline replay of fixed hard calls should gate each release, and online monitoring of speech-to-text confidence, latency and containment should catch the drift that a fixed clip set cannot contain.

---

Testing voice agent turn detection and barge-in means replaying audio you have labelled in advance and counting how often the agent's behavior matched the label. This article covers how to build that test set, which two quantities to count, how to read the failures and where to run the tests. It is about measurement only; it does not recommend settings, and the architecture and handoff decisions are in the [pillar guide to running a voice AI agent in production](/articles/voice-ai-agent-in-production). The reference material is [evaluating voice agents](/resources/evaluating-voice-agents) and [voice and realtime agents](/resources/voice-realtime-agents).

## What does the corpus say about testing barge-in?

The corpus ranks interruption handling ahead of latency tuning, because the voice-realtime-agents reference (updated 2026-09-28) says to handle interruptions before tuning latency and calls mishandling barge-in "the most common CSAT failure in voice agents". It defines barge-in as detecting the user speaking while the agent responds, flushing the text-to-speech output buffer and restarting the listening stage. It also says model-based end-of-turn detection is lower in latency than silence timeouts, with no figure attached.

The evaluating-voice-agents reference (updated 2026-09-15) supplies the metrics. It lists, among end-to-end measures, barge-in accuracy, defined as correctly telling a genuine interruption from a backchannel apart, and false-interruption rate. It states no threshold for either and no way to compute them, so everything below on how to count is this article's suggestion.

## How do you build a test set for turn detection and barge-in?

Build the test set from short audio clips that each carry one label saying what the agent should have done. This is the author's suggestion, not a corpus method. A workable label scheme has three classes:

| Label | What the clip contains | Correct agent behavior |
|---|---|---|
| Genuine interruption | The caller speaks over the agent to redirect, correct or stop it | Agent stops talking and listens |
| Backchannel | A short acknowledgement such as "uh-huh" or "right" while the agent speaks | Agent keeps talking |
| Silence or noise | Background sound, a cough, a door, or line noise with no caller intent | Agent keeps talking |

Cover the conditions the evaluating-voice-agents reference says an offline set should include on purpose: accents, background noise and crosstalk. VoiceBench, which that reference describes as the first benchmark built for LLM-based voice assistants, varies accent, background noise and speaking rate, which is a useful checklist for the axes to vary.

Have at least two people label each clip independently and resolve disagreements before use. Where people disagree about whether a "yeah, but" is an interruption, the policy is undefined, not the agent wrong, which leads to the next section.

## Who decides what counts as a backchannel?

The team that owns the agent decides, because the corpus gives the principle (distinguish genuine interruptions from backchannels) but no list of which utterances fall on which side. Decide the policy in writing before labelling, for example how to treat "okay", "wait", "no" or a caller speaking a number while the agent reads one back. Without a written policy, two labellers produce two test sets and any score is unrepeatable.

Revisit the policy when production logs show callers using phrases the set does not contain, and version the labelled set so a score from March can be compared with one from October.

## How do you compute barge-in accuracy and false-interruption rate?

Compute both as simple count ratios over the labelled set. These definitions are this article's formulation, since the corpus names the quantities without a recipe.

- **Barge-in accuracy** is the number of clips where the agent's behavior matched the label (yielded on a genuine interruption, kept talking on a backchannel) divided by the number of interruption and backchannel clips.
- **False-interruption rate** is the number of backchannel and noise clips on which the agent stopped talking divided by the number of backchannel and noise clips.

```bash
# results.csv columns: clip_id,label,agent_stopped   (label: interruption|backchannel|noise; agent_stopped: 1|0)
awk -F, 'NR>1 && ($2=="noise"||$2=="backchannel") {n++; if ($3==1) f++}
  END {printf "false_interruption_rate=%d/%d\n", f, n}' results.csv
```

Report the raw counts next to the ratio. A rate computed on a small set moves a lot when one clip flips, and the corpus supplies no pass line, so decide your own and record why.

## What failure classes should a barge-in test separate?

A barge-in test should separate cut-off failures from talk-over failures, because they are opposite errors that one averaged score blends together. The corpus names both ends: the evaluating-voice-agents reference describes a turn-taking model "that talks over the caller", and the voice-realtime-agents reference warns about cutting off mid-sentence unnecessarily.

| Failure class | What the caller experiences | Which label exposes it |
|---|---|---|
| Cut-off | The agent stops or replies on a backchannel or noise | Backchannel and noise clips |
| Talk-over | The agent keeps speaking after the caller has clearly interrupted | Genuine-interruption clips |

The corpus gives no prevalence for either, so do not assume the first is rarer. Tune nothing on the strength of this article; the point is only that a release which fixes one can silently worsen the other, and per-class counts show it.

## Where can you test: cascaded seam or black box?

In a cascaded pipeline you can test the barge-in seam directly, and in a native speech-to-speech model you can only test from outside. The voice-realtime-agents reference says explicit barge-in logic is needed at each stage boundary in a cascade, and that native models handle interruption inside the model.

For a cascade, assert on the flush: after a genuine interruption, check that the queued synthesized audio was discarded and the listening stage restarted, instead of only observing silence at the speaker. That separates a turn-detection fault from a playback-buffer fault. For a native model, play the clips in, record the output audio and judge whether and when it stopped. As inference, that is a weaker diagnosis: a wrong result tells you the model misbehaved but not why.

## Should turn detection tests run offline or online?

Run them in both places, since the evaluating-voice-agents reference gives them different jobs. Offline evaluation replays a fixed set of recorded or synthetic calls, including deliberately hard ones, before every release; it is fast and reproducible but blind to conditions the set does not contain. Online evaluation monitors STT confidence, per-stage and end-to-end latency and containment rate in production. The reference's rule is to gate a release offline and catch what it misses online.

For reliability across repeated runs, the same reference notes that τ³-bench added a full-duplex voice mode scored with the pass^k metric. Per the evaluating-ai-agents reference, pass^k is the probability that all k independent trials succeed, so a clip set replayed several times exposes agents that behave inconsistently on identical audio. For drift alerts on the online side, see [online agent metrics and drift monitoring](/articles/online-agent-metrics-and-drift-monitoring); for the delay numbers that sit beside these tests, see the [voice agent latency budget](/articles/voice-agent-latency-budget).

## What should you log on every turn?

Log enough per turn to rebuild any disputed interruption decision after the fact. This field list is the author's suggestion:

- The turn identifier and the audio span, stored or referenced so it can be replayed.
- Whether the agent was speaking when caller audio arrived, and whether it stopped.
- The transcript, with STT confidence where the vendor exposes it.
- The end-of-turn decision and the timestamp of the caller's last speech according to your own detector.
- In a cascade, whether the TTS buffer was flushed.
- The label a human later assigns, if the turn is sampled for review.

Sampled, human-labelled production turns are how the offline set stays honest: each real misjudged turn becomes a new clip with a label.

## What can ChangeGamer say from its own data?

ChangeGamer has no barge-in measurements of its own, because it operates no voice agent and handles no calls. As of 2026-10-08, its evaluating-voice-agents reference names barge-in accuracy and false-interruption rate but states no threshold and no computation recipe. That is why the ratios above are labelled as this article's formulation rather than a standard. Anything here that is not attributed to a reference is suggestion or inference, and should be validated on your own recorded calls.

## Frequently asked questions

### What is barge-in in a voice agent?

Barge-in is the agent detecting that the caller has started speaking while it is still talking, flushing its pending synthesized audio and going back to listening. The voice-realtime-agents reference describes it exactly that way, and calls getting it wrong the most common CSAT failure in voice agents.

### What is the difference between a barge-in and a backchannel?

A barge-in is a genuine interruption the agent should yield to, while a backchannel is a short acknowledgement such as "uh-huh" or "right" that the caller makes. The voice-realtime-agents reference says to tell them apart so the agent does not cut itself off mid-sentence unnecessarily.

### How do you measure false interruptions in a voice agent?

The evaluating-voice-agents reference names false-interruption rate but gives no formula. One workable definition, which is this article's own and not a corpus recipe, is the number of backchannel or noise clips on which the agent stopped talking divided by the total number of such clips in the labelled set.

### Can you test barge-in on a native speech-to-speech model?

Yes, though mostly from outside: you play audio into the model and observe whether and when the output stops, because the voice-realtime-agents reference says turn detection and interruption handling live inside the model. There is no buffer-flush seam to inspect, so black-box replay and production logs are the available evidence.


---

## The rest of this guide

- [Voice AI Agent in Production: Architecture, Latency, Handoff and Launch](https://changegamer.ai/articles/voice-ai-agent-in-production.md): How to build and run a voice AI agent in production: cascaded vs speech-to-speech architecture, turn detection and barge-in, the latency budget, tool calls in a voice loop, transport, cost, human handoff and launch sequencing.
- [Voice Agent Latency Budget: Where the 600 ms Goes and How to Hold It](https://changegamer.ai/articles/voice-agent-latency-budget.md): How to split a 600 ms voice agent latency budget across speech-to-text, the language model, speech synthesis and tool calls, how to measure each stage, and which levers hold the budget.
- [What to Log and Filter in a Voice Agent, by Architecture](https://changegamer.ai/articles/voice-agent-architecture-cascaded-vs-speech-to-speech.md): What to log and filter in a voice agent depending on architecture: where a cascaded pipeline gives you text seams for logging, redaction and output filters, what a native speech-to-speech model leaves you to do from outside, and what a model swap forces you to re-test.
- [Function Calling in a Voice Agent: What a Tool May Be in a Spoken Loop](https://changegamer.ai/articles/tools-in-a-voice-agent-loop.md): Function calling in a voice agent: which vendors and frameworks support tools as of the corpus's 2026-07-09 verification, how to classify tools as lookup, action or handoff, what barge-in does to a side effect, and a checklist for admitting a tool.
- [Voice Agent Transport: WebRTC vs WebSocket, and Where SIP Fits](https://changegamer.ai/articles/voice-agent-transport-webrtc-vs-websocket-vs-sip.md): WebRTC vs WebSocket for voice AI agents: which transport each vendor documents as of the corpus's 2026-07-09 verification, how cascaded stacks use WebSocket components, a decision tree by client type, and a per-transport test checklist.
- [Voice Agent Human Handoff: What Happens on the Call When the Agent Lets Go](https://changegamer.ai/articles/voice-agent-human-handoff.md): Voice agent human handoff mechanics: the handoff as a tool that ends the agent's session, barge-in during the transfer announcement, what the caller hears on hold and on failure, a context payload schema, and a real-call test checklist.
- [How to Compute Your Voice Agent Cost per Minute](https://changegamer.ai/articles/voice-agent-cost-per-minute.md): How to compute a voice agent cost per minute from your own pilot: symbolic formulas for native speech-to-speech and cascaded STT, LLM and TTS, the variables that move the result, idle audio, retries, tool-call turns, and a sensitivity check.
- [Voice Agent Guardrails: Checking What the Agent Says Before the Caller Hears It](https://changegamer.ai/articles/guardrails-for-spoken-output-in-voice-agents.md): Voice agent guardrails for spoken output: sentence-level gating before TTS, fail-closed judges, spoken numbers a text PII filter misses, readback before state changes, transcription-driven injection, refusal phrasing for the ear, and an adversarial test list.
- [How to Monitor a Voice Agent in Production: Dashboard, Alerts and Runbook](https://changegamer.ai/articles/voice-agent-online-monitoring.md): How to monitor a voice agent in production: per-stage latency percentiles, transcription confidence, containment paired with a resolution signal, live false-interruption rate, a handoff trigger review, and call-audio retention, each with what to chart, alert on and do.
- [How to Handle Voice AI Vendor Model Changes Without Regressing Callers](https://changegamer.ai/articles/voice-agent-vendor-change-management.md): How to handle voice AI vendor model changes: classify the change, record the model id on every trace, replay an offline set with hard calls on old and new, canary before cutover, keep a rollback, and run a deprecation-readiness routine.
- [Voice Agent Launch Checklist: The Go/No-Go Gates Before Real Callers](https://changegamer.ai/articles/voice-agent-launch-checklist.md): A voice agent launch checklist as yes/no gates: scope, offline replay, per-stage latency, barge-in, tools, transport, handoff, cost, guardrails, monitoring and rollback, each with a pass signal and the article that owns the method.
- [How Does a Voice Agent Handle Callers Who Speak Another Language?](https://changegamer.ai/articles/voice-agent-multilingual-callers.md): How a live voice agent handles callers who speak another language: detecting the language, choosing speech components per language, slicing evaluation by language and handing off when the language is unsupported, with the thin corpus evidence stated plainly.

## Reference resources

- https://changegamer.ai/resources/voice-realtime-agents.md
- https://changegamer.ai/resources/evaluating-voice-agents.md
- https://changegamer.ai/resources/evaluating-ai-agents.md

All guides: https://changegamer.ai/api/articles.json · Reference corpus: https://changegamer.ai/llms.txt
Licensing: https://changegamer.ai/api/pricing.json (offer catalog) · https://changegamer.ai/api/payment.json (payment methods, HTTP 402 flow) · access guide: https://changegamer.ai/resources/access-and-pricing.md