Turn-taking in voice AI is how a voice agent decides that the caller has finished speaking and when to start its reply. It combines end-of-turn detection, often called endpointing, with the time the agent needs to produce an answer.
People switch turns quickly. A study of 10 languages led by Tanya Stivers (PNAS, 2009) timed answers to yes/no questions. On average, the answer began 208 milliseconds after the question ended. Every language showed the same pattern of avoiding overlap and keeping silences short. A voice agent, or voicebot, fits speech recognition, a language model, and speech synthesis into each gap, so its pauses usually run longer.
How end-of-turn detection works
Silence timeout. The simplest method waits for a fixed stretch of silence after the caller's last word. VoiceXML 2.0 puts reasonable values at 0.3 to 1.0 seconds. Twilio's 2025 guide to voice agent latency says most defaults sit around 500 milliseconds. A long timeout makes the agent slow, and a short one cuts callers off mid-sentence.
A longer timeout for unfinished input. VoiceXML also defines a separate, usually longer timeout for speech that looks incomplete, so callers can pause mid-utterance, for example to breathe. Many voice agents apply the same idea when a caller reads out a long number in groups.
Model-based endpointing. Newer systems use a trained model to judge whether the turn is complete. Twilio notes that this allows tighter thresholds and can also cause sudden jumps in the slowest response times.
Interruptions. When the caller speaks while the agent is talking, barge-in rules decide whether the agent stops.
Where the time goes
The gap a caller hears runs from the end of their speech to the moment the reply reaches their ear. Twilio calls this the mouth-to-ear turn gap. Its November 2025 guide, which is vendor guidance, sets median starting targets of 350 milliseconds for speech-to-text and 375 for the model's first token. The first audio of the reply has a target of 100 milliseconds.
The full mouth-to-ear target in the same guide is 1,115 milliseconds, with 1,400 as an upper limit. Twilio describes these as starting benchmarks. It names end-of-turn detection as often the largest delay, because systems are tuned to avoid cutting callers off.
Back-end lookups add their own time when the agent checks an account or a balance mid-call. Averages hide slow turns, so it helps to track the gap at the median and at the 95th percentile.
Tuning turn-taking for a contact center
Different timeouts for different steps. A yes-or-no question can use a short timeout. Reading out a card number, a date, or an address needs a longer one.
A short acknowledgment during slow steps. Some agents say "One moment, let me check" before a long lookup, so the caller does not fill the silence with "Hello?".
Noisy lines. Background speech can keep the silence from ever arriving, so the agent waits too long. It helps to test turn-taking on recordings from cars, shops, and speakerphones.
Measures from real calls. Useful counts include the turn gap at the median and 95th percentile, and how often the agent cuts in on the caller. Callers repeating themselves, or asking whether anyone is there, are worth counting too.
Frequently asked questions
What is endpointing in speech recognition?
Endpointing is the decision that a speaker has finished an utterance. The recognizer then finalizes the transcript, and the agent can respond. It usually relies on a silence timeout, a trained model, or both.
What is a good response time for an AI voice agent?
We found no formal standard. Twilio's vendor guidance of November 2025 sets a median target of 1,115 milliseconds, from the end of the caller's speech to the reply reaching them. A contract value is best set from your own baseline, at the median and the 95th percentile (see service level agreement).
Why does a voice agent interrupt callers?
Often because its end-of-turn timeout is too short for the moment. Callers pause to find a card, to think, or between groups of digits. A model-based endpointer can also misjudge an unfinished sentence. Longer timeouts on data-collection steps, and a read-back before acting, reduce the problem.
How is turn-taking different from latency?
Latency is the time the agent takes to produce its reply. Turn-taking also covers when the agent decides to reply and how it handles overlap. A fast agent with a poor end-of-turn decision still cuts people off.
How we approach this at Zowie
Zowie's AI agents work on voice, chat, and email, with over 100 million conversations a year across our clients. On a call, the language model speaks. A separate, deterministic Decision Engine returns every amount, rate, offer, disclosure, and escalation from your rules, with an audit log. To hear it on your own call types, you are welcome to book a demo.
Explore: Barge-in, Word error rate, Conversational IVR, Voice AI, Service level agreement, SIP trunking