AI glossary · Models
Inference
In one sentence
Inference is when an already-trained AI model is put to use: it receives a new input, such as a question, an audio clip or an image, and computes a response from what it learned, without changing its parameters.
Also known as: AI inference · model inference · inferencing · prediction
What is inference?
An AI model has two stages in its life. In training, it learns. In inference, it works. Every time an assistant answers you, a call gets transcribed or a scanned document turns into text, there’s an inference behind it.
Put it to work
UTIO's AI, through an API
From €29/mo + VAT
For developers and agencies building their own products.
See the AI APIThe word comes from statistics: the model “infers” an answer from the patterns it learned. During inference it learns nothing new; its parameters stay fixed. If it seems to “remember” earlier parts of a conversation, that’s because the information is sent back in its context with each request.
Related to this term
AI API
- Transcription, voice, text, image, OCR and embeddings
- Per-client keys with a spending cap
- Prepaid wallet: €50 minimum top-up
From €29/mo + VAT Panel account, plus usage
See the AI APIHow it works
- Input. The request arrives: text, audio or an image, converted into numbers. For text models, into tokens.
- Computation. The data passes through the model layer by layer, using its parameters. A language model repeats this step to produce its reply one token at a time.
- Output. The result is turned back into something useful: text, audio, a label or structured data.
Two things matter most at inference time. Latency, or how long the answer takes, which is critical on a phone call. And cost, which depends on model size, how many tokens go in and out, and the hardware. To bring it down, people use smaller models, techniques such as quantisation, or pick a different model depending on how hard each task is.
Example
A building and renovation firm in Leicester uses AI in two ways. Overnight it transcribes and summarises every call from the day: there’s no rush, so a slower, cheaper model is fine. During the day an assistant answers the phone live: every second counts, so it needs fast inference.
The same kind of task, “understand language”, has different needs depending on when it runs. Choosing the right model for each case is mostly an inference decision: a balance between quality, speed and cost.
Inference at UTIO
With UTIO’s AI API, what you pay for is inference: each transcription, text response, image or document run through OCR. A single key unlocks every capability, and the per-unit rates, by model tier (basic, standard or superior), are published in the panel. That lets you use a higher tier where quality matters most and the basic tier for simple, high-volume jobs.
The agents work differently: with the voice agent, the minute already includes the model, the voice, transcription and telephony, so there are no inferences for you to count. What you do notice is latency: on a call, the agent has to reply without awkward silences.
Frequently asked questions
What's the difference between inference and training?
In training, the model learns: its parameters are adjusted using lots of examples. In inference, the model works: it responds to new inputs with fixed parameters. Training happens rarely and costs a lot up front; inference happens on every use and its cost grows with volume.
Why is inference on a large model slower and more expensive?
Because every response involves calculations across all or much of its parameters, and language models repeat that for every token they generate. More parameters and longer answers mean more compute, more time and more cost.
Where does inference run?
Usually on the provider's servers, which you call through an API. Smaller models can also run on your own server, a laptop or a phone, which gives you more control over data but needs suitable hardware.
Related terms
- Model training
Model training is the process in which an AI model learns from a dataset: it's shown examples, its errors are measured and its parameters are adjusted, over and over, until its outputs are good.
- Latency
Latency is the time between making a request and starting to receive the response. In a voice agent, it's the silence from the moment the caller stops talking to the moment they hear the agent's first word, and it decides whether the call sounds natural.
- Token
A token is the chunk of text a language model works with: a word, part of a word or a punctuation mark. Models read, count and bill in tokens, not in words or characters.
- AI model
An AI model is the program you get by training an algorithm on data: a set of learned parameters that, given a new input, produces an output such as a prediction, a label, a piece of text or an image.
- Quantization
Quantization is a technique that stores an AI model's parameters as lower-precision numbers, for example 8 or 4 bits instead of 16. The model takes up less memory and often runs faster, in exchange for a loss of quality that can be small.
Updated on