Transformer

What is a transformer?

Before transformers, language models read text one word at a time, left to right, carrying a running summary of what they’d seen. In long sentences, the beginning faded away. And because the work was sequential, training on large amounts of data was slow.

The transformer fixed both problems. It looks at the whole text at once and, for every word, calculates how much attention to pay to each of the others. That calculation is called attention. In “The customer rang the garage because her car wouldn’t start”, attention helps link “her” to “the customer” rather than “the garage”.

Because every position is processed in parallel, transformers can be trained on enormous volumes of text using graphics cards. That’s where today’s LLMs and families like GPT come from.

How it works

In simplified terms, a transformer:

1. Converts each token into a vector of numbers and adds information about its position in the sentence.
2. Applies attention: each token looks at all the others and takes in whatever is relevant. This happens several times in parallel (“multi-head attention”) so it can capture different kinds of relationship at once.
3. Repeats the process across many stacked layers. Early layers pick up simple relationships; later ones capture more abstract meaning.
4. Outputs a probability for every possible next token.

The original design had two halves, an encoder and a decoder, built for translation. Today’s generative models usually keep only the decoder. Variants such as mixture of experts change part of the block to make it more efficient.

Example

A travel agency in Edinburgh gets enquiries like: “I’d like to take my mum to Rome in May, but she can’t walk very far, which hotel would you suggest?” To answer well, the system has to connect “she” with “my mum”, and “can’t walk very far” with the choice of hotel, even though they sit apart in the sentence.

Linking distant parts of a text is exactly what transformer attention does. That’s why a modern assistant can follow long, rambling requests that a keyword-based system would miss.

How it affects your business

You don’t need to know how to build a transformer to use AI. Two practical consequences are worth knowing, though. First, the cost of attention grows with the length of the text, which is why models have a limited context window and why techniques like RAG send only what’s needed. Second, the model answers from the connections it finds in the text, not from verified facts, which is where hallucinations come from.

When a UTIO AI agent makes sense of a long message from your customer on WhatsApp, a transformer is doing that work under the hood.