Jailbreak

What is a jailbreak?

The term comes from phones: “jailbreaking” an iPhone meant removing the manufacturer’s restrictions. In AI it means something similar: getting a model to do what its rules don’t allow.

Language models are trained to refuse certain requests, such as giving dangerous instructions or producing harmful content. On top of that, the company using the model adds its own rules in the system prompt: don’t discuss competitors, don’t give bespoke prices, stay on topic. A jailbreak tries to get past any of those layers.

It’s not the same as prompt injection. A jailbreak goes after the model’s rules; injection tries to make an application follow someone else’s orders, often hidden in a document. In practice, many attacks mix the two.

How it works

We won’t give recipes here, but the general idea is well known. Attempts tend to disguise the forbidden request so the model doesn’t recognise it, or put pressure on it to rank “being helpful” above its rules. They work because the model doesn’t understand rules the way a person does: it learns patterns, and there are always combinations its training didn’t cover.

Model makers close known methods with each release, but new ones appear. So defence can’t rely on the model alone:

1. Don’t treat instructions as the only barrier. Anything that must never happen is blocked with permissions and code.
2. Least privilege. If the assistant can’t see other customers’ data or issue refunds, tricking it achieves little.
3. Human review of anything sensitive.
4. Filter what goes out before it’s sent.
5. Test your own system with attempts to trick it before opening it to the public.

All of this is part of your guardrails.

Example

A car dealership in Birmingham puts an assistant on its website to book workshop appointments. Someone spends several messages trying to talk it into confirming in writing a 50% discount on a car. They get a strange sentence and nothing more: the assistant has no access to prices and can’t issue offers. All it can do is book a workshop slot and pass the conversation to a salesperson.

The weakness exists, but the damage is minimal. That’s what a well-designed system looks like.

What it means for your business

If you put an AI agent in front of the public, someone will try to knock it out of its role. Rather than hunting for a model that can’t be tricked, limit what the agent can do.

At UTIO, the agent’s autonomy is gradual and starts in draft mode. You decide which tasks it handles on its own and when it passes the conversation to a person: by time of day, by topic or when the customer asks.