Prompt injection

What is prompt injection?

A language model can’t reliably tell “my company’s instructions” apart from “the text I’ve been asked to read”. To the model, it’s all text. If something in that text looks like an order, it may end up following it.

That’s prompt injection. It comes in two forms:

• Direct: the user types instructions into the chat to change how the assistant behaves.
• Indirect: the instructions are hidden in content the agent reads on its own, such as an email, a PDF or a web page. This is the more dangerous kind, because the attacker never even talks to the agent.

The name echoes SQL injection, a classic attack on databases. The difference is that here there’s no watertight way to separate data from instructions.

How it works

The risk depends on what the agent can do. An assistant that only answers questions can, at worst, say something inappropriate. An AI agent that reads email and can also send messages, look things up in the CRM or change records is another matter: a planted instruction could make it act on your behalf.

The defences that work are about the system, not the wording:

1. Least privilege. The agent has only the permissions it really needs.
2. Human confirmation for sensitive actions: payments, bulk sends, deletions, other people’s data.
3. Keep data apart from instructions. Outside content is marked as reference material, never as orders.
4. Filter inputs and outputs. Flag suspicious content and check what comes out before acting on it.
5. Nothing secret in the system prompt: no keys, no passwords.
6. Logging of what the agent does, so problems can be spotted and fixed.

All of these are part of the system’s guardrails.

Example

An accountancy practice in Belfast trials an agent that reads incoming email, summarises it and drafts replies. One day an email arrives from an unknown sender with hidden text addressed “to the assistant”, asking it to forward client documents to another address.

The agent has no permission to forward documents or send anything without review: it only prepares drafts. The email ends up as a summary, which the accountant reads and flags as suspicious. The attack goes nowhere, because the agent couldn’t do what it was being asked.

What it means for your business

Before giving an agent access to your email, your CRM or your documents, ask what it can do without anyone checking. That’s the real measure of the risk.

At UTIO, autonomy is gradual: no AI, ask first, draft or automatic, and the agent starts in draft. The email agent quarantines anything suspicious or irrelevant and leaves its replies as drafts in the CRM. Moving up a level is your decision, one task at a time.