UTIO
☎️ Try the demoSign up

AI glossary · Prompts

Context window

In one sentence

A context window is the maximum amount of text, measured in tokens, that a language model can take into account at once: instructions, conversation, documents and its own reply. Anything outside it, the model simply doesn't see.

Also known as: context length · context size · token limit · model working memory

What fits in the context window The context window is a fixed-size space, for example 128,000 tokens, that the model reads in one go. It holds the system prompt instructions, the recent conversation history, the retrieved documents and the current question. When everything doesn't fit, the oldest messages drop out and the model can no longer see them. Oldest messages don't fit: the model no longer sees them drop out Fixed size (e.g. 128,000 tokens) Instructions system prompt Recent history latest messages in the chat Documents retrieved passages (RAG) Current question the customer's latest message
Everything the model “takes into account” for an answer has to fit in its window. Whatever falls out no longer exists for it, which is why platforms summarise or trim the oldest part.

What is a context window?

Picture a desk. Everything the model needs to answer has to be on it: the instructions, the conversation so far, the reference documents and space to write the reply. The context window is the size of that desk, and it’s measured in tokens.

Put it to work

AI agents that look after your customers

From €49/mo + VAT

Voice, WhatsApp and email with the phone line included: agent, numbers and CRM in one fee.

See plans and pricing

Some models work with a few thousand tokens; others accept hundreds of thousands or more. The principle doesn’t change: whatever doesn’t fit doesn’t exist for the model when it answers.

Related to this term

UTIO agents

  • Available 24/7 in 26 languages
  • Set it up yourself for €0: built from your website
  • 14-day free trial, no card needed

From €49/mo + VAT

See plans and pricing

How it works

For each reply, the application assembles a bundle and sends it to the model:

  1. The system prompt with the fixed rules.
  2. The conversation history.
  3. Any document extracts that are needed.
  4. The user’s new message.

The model’s reply takes up space too. When the bundle nears the limit, something has to go: the oldest messages, a summary in their place, or the least useful documents.

That’s why well-built systems don’t send a company’s entire documentation with every question. With RAG, they search for the relevant passages and send only those. It costs less, answers come back faster and the model gets less distracted.

One practical point: the model keeps nothing from one reply to the next. It reads the whole desk from scratch each time. If it seems to “remember”, that’s because the application sends the history back in.

Example

A payroll bureau in Manchester trials an assistant that answers questions about payslips. At first, the team pastes the full employment handbook into every question. It works, but each answer is slower and more expensive. And in long conversations, the assistant starts forgetting what the client said at the start, because the history no longer fits alongside the handbook.

The fix is to split the handbook into sections and search only the ones relevant to each question. The desk stays clear for the conversation.

What it means for your business

If you use an AI agent, the context window explains two things you’ll notice. First, why your documentation should be tidy and free of duplication, so the right passage gets found. Second, why in a very long conversation the agent may lose details from the beginning.

At UTIO, agents answer from your knowledge base: they search the documentation you load rather than sending all of it. And the key points of each conversation are logged in the CRM, which is stored in Spain. You’ll find this in the support vertical, for example.

Frequently asked questions

What happens when you go over the context window?

It depends on the application. Some drop the oldest part of the conversation, some summarise it and some return an error. Either way, the model loses part of the information and may answer as if it never had it.

Is a bigger context window always better?

It gives you room, but it comes at a cost. Every token you send is paid for and adds to response time. And with very long contexts, models make poorer use of information buried in the middle. Sending only what's relevant usually works better.

Is the context window the model's memory?

It's the model's working memory for a single reply. The model remembers nothing between conversations: if an agent remembers a customer, it's because the application stores that data and passes it back in.

Related terms

  • Token

    A token is the chunk of text a language model works with: a word, part of a word or a punctuation mark. Models read, count and bill in tokens, not in words or characters.

  • Prompt

    A prompt is the instruction or text you give an AI model to get something done: a question, a command, a document to summarise or a mix of all three. Much of what the model gives back depends on the prompt.

  • RAG

    RAG (retrieval-augmented generation) is a technique in which a language model first retrieves the relevant passages from your own documents and then writes its answer from them, instead of relying on what it memorised in training.

  • Agent memory

    Agent memory is the information an AI agent keeps so it can use it later: what has been said in the current conversation (short-term) and stored details from earlier conversations, such as a customer's preferences or history (long-term).

  • LLM

    An LLM (large language model) is an AI model trained on vast amounts of text to predict the next word. That single skill lets it draft, summarise, translate, classify and hold a conversation.

Updated on

← Back to the glossary

UTIO is the AI agent line of Universal Telecom, an operator registered with the CNMC.