AI glossary · Voice
Latency
In one sentence
Latency is the time between making a request and starting to receive the response. In a voice agent, it's the silence from the moment the caller stops talking to the moment they hear the agent's first word, and it decides whether the call sounds natural.
Also known as: response time · delay · lag · time to first token
What is latency?
In any computer system, latency is the wait. In AI, the most common measure is time to first token: how long the model takes to start answering. In a web chat, a few seconds is fine. On a phone call it isn’t: in normal conversation people hand over the turn in a fraction of a second, and a long silence makes the caller think the line has dropped or they weren’t understood.
Put it to work
A voice agent that picks up the phone
From €149/mo + VAT
It answers calls 24/7, books appointments and logs every call in the CRM.
See the voice agentSo in voice agents, latency isn’t a technical footnote. It’s part of the experience.
Related to this term
UTIO Voice
- 500 minutes and a number included
- Natural conversation in 26 languages
- Introduces itself as a virtual assistant
- 14-day free trial, no card needed
From €149/mo + VAT
See the voice agentHow it works
In a voicebot, each reply passes through a chain of steps, and each one adds time:
| Step | What happens | What it adds |
|---|---|---|
| End-of-turn detection | Knowing the caller has finished speaking | A cautious pause so as not to cut in |
| Speech to text | Turning speech into text | Transcription time |
| Language model | Working out the reply, sometimes calling a tool | Time to first token |
| Text to speech | Turning text into audio | Time to first sound |
| Network and telephony | Carrying the audio | Network round trips |
The techniques for reducing it are well known:
- Streaming end to end. Don’t wait for complete sentences: transcribe while the caller is talking and start speaking as soon as the model produces its first words.
- Right-sized models. A huge model for a simple question adds waiting without improving the answer.
- Fewer hops. Every extra provider in the chain adds another network trip.
- Handling interruptions. If the caller talks over the agent, it should stop and listen, as a person would.
Example
A car garage in Nottingham trials two voice agents for booking services. With the first, there’s a clear pause after every question. Callers repeat themselves or say “hello?”, and the conversation gets tangled. With the second, which streams its replies and can be interrupted, the call flows and the booking is done in fewer turns. Both use similar models; the difference is in how the pieces are put together.
Latency at UTIO
In UTIO’s voice agent, transcription and the language model run on UTIO’s own servers, and the telephony runs on its own carrier network, so the call doesn’t bounce between lots of providers. The agent understands ordinary speech, interruptions and accents, and nobody has to press keys or speak slowly.
The best way to judge latency is to listen to a call. During the free trial you can ring your own agent and decide whether the pace feels natural.
Frequently asked questions
Are latency and speed the same thing?
Not quite. Latency measures how long until the first piece of the answer arrives. Speed, or throughput, measures how much content arrives per second once it starts. In voice, latency matters most: a long pause at the start is far more noticeable than a sentence that arrives at a steady pace.
Why do bigger models often have more latency?
Because every word they generate takes more computation. That's why voice systems often use lighter models, or combine them: a fast one for the conversation and a more capable one only when real reasoning is needed.
Does latency depend on my connection?
Partly. On top of the models' processing time comes the trip the audio makes across the network and the phone system. The closer the servers and the fewer hand-offs between providers, the less delay builds up.
Related terms
- Voicebot
A voicebot is a program that holds spoken conversations: it listens to what a person says, works out what they want and answers out loud. It can run on a phone line, a website, an app or a smart speaker, and with generative AI it handles open conversation, not just menus.
- Speech to text
Speech to text (STT) is the technology that turns spoken audio into written text. It's the software component, usually a model reached through an API, that does the listening in a voicebot, a dictation tool or a transcription service, either live or on recordings.
- Text to speech
Text to speech (TTS) is the technology that turns written text into spoken audio. It's the component, usually a model reached through an API, that gives a voice to a phone agent, a screen reader or any audio generated from text.
- Inference
Inference is when an already-trained AI model is put to use: it receives a new input, such as a question, an audio clip or an image, and computes a response from what it learned, without changing its parameters.
- Token
A token is the chunk of text a language model works with: a word, part of a word or a punctuation mark. Models read, count and bill in tokens, not in words or characters.
Updated on