AI glossary · Models
Small language model (SLM)
In one sentence
A small language model (SLM) is a language model with far fewer parameters than an LLM. It knows less about everything, but it replies faster, costs less and can run on a modest server, a laptop or even a phone.
Also known as: SLM · small LLM · lightweight model · compact language model
What is a small language model?
An SLM is the compact version of a language model. It works just like an LLM, predicting tokens one after another with a transformer network. The difference is scale. With fewer parameters it holds less general knowledge and struggles more with long chains of reasoning, but it needs far less memory and computing power.
Put it to work
UTIO's AI, through an API
From €29/mo + VAT
For developers and agencies building their own products.
See the AI APIThat trade-off makes it useful in three situations: when speed matters (a voice conversation can’t tolerate long pauses), when volumes are high and cost per request adds up, and when data mustn’t leave a particular machine.
Several model families release small versions alongside their large ones, such as Phi, Gemma, Llama and Mistral. Many are distributed as open-source AI or with open weights.
Related to this term
AI API
- Transcription, voice, text, image, OCR and embeddings
- Per-client keys with a spending cap
- Prepaid wallet: €50 minimum top-up
From €29/mo + VAT Panel account, plus usage
See the AI APIHow it works
There are several ways to get an SLM, often used together:
- Train a small model from scratch on carefully selected data, favouring quality over quantity.
- Distil a large model: the small one learns to imitate the large one’s answers.
- Compress it afterwards with quantization, which lowers the precision of its numbers so it takes up less space.
It can then be specialised through fine-tuning. An SLM tuned to sort emails in one sector can come close to a general-purpose LLM on that task, at a fraction of the cost.
Example
An electrical wholesaler in Birmingham receives hundreds of orders by email every day, each one written differently. It needs to pull out the product code, quantity and delivery address to feed them into its ERP.
You don’t need the biggest model on the market for that. A well-instructed SLM extracts those fields in milliseconds and at very low cost. Doubtful orders, such as one quoting a product code that doesn’t exist, get flagged for a member of staff to check.
Small language models at UTIO
UTIO’s AI API lets you choose the model tier for text generation: basic, standard or superior. The thinking is the same as behind SLMs: not every task needs the most capable model. For classifying, summarising or extracting data at volume, the basic tier is often enough; for long or sensitive replies, it’s worth stepping up.
Per-unit rates for each tier are published in the panel, along with the model reference. Picking the right model size is one of the decisions that most affects the running cost of an AI agent.
Frequently asked questions
How small does a model have to be to count as an SLM?
There's no official cut-off. SLM usually refers to models with a few billion parameters or fewer, as opposed to LLMs with tens or hundreds of billions. The line keeps shifting: what counts as small today would have been huge a few years ago.
Is an SLM good enough for a business chatbot?
For well-defined tasks, yes: sorting messages, extracting order details or answering questions about a catalogue with the help of RAG. For open-ended conversations and multi-step reasoning, a larger model is usually more reliable.
Can I run an SLM on my own hardware?
Many SLMs are released with open weights and fit on a computer with a decent graphics card, especially once quantized. Your data then stays on your own infrastructure, as long as you're willing to maintain it.
Related terms
- LLM
An LLM (large language model) is an AI model trained on vast amounts of text to predict the next word. That single skill lets it draft, summarise, translate, classify and hold a conversation.
- Language model
A language model is an AI system that calculates which word, or token, is most likely to come next in a piece of text. By repeating that prediction it can complete sentences, answer questions, summarise or translate.
- Quantization
Quantization is a technique that stores an AI model's parameters as lower-precision numbers, for example 8 or 4 bits instead of 16. The model takes up less memory and often runs faster, in exchange for a loss of quality that can be small.
- Inference
Inference is when an already-trained AI model is put to use: it receives a new input, such as a question, an audio clip or an image, and computes a response from what it learned, without changing its parameters.
- Fine-tuning
Fine-tuning means further training an AI model that's already been pre-trained, using a small set of your own examples, so it specialises in a particular task, format or style without starting from scratch.
Updated on