We break our own agents
and publish what happened.
Three sandboxed demonstrations of an AI agent being talked into something nobody authorised — then the same agent, same prompt, same files, refusing. Every figure on these pages comes from the runner output, and the suite is public so you can run it yourself.
Read this before the numbers
The agent in these labs is stub/document-follower — a scripted agent built to follow tool-shaped instructions it finds in documents. It is not a commercial model, and it is not a stand-in for one. It breaches on every control run because that is what it was written to do.
So the control column measures the harness, not the industry. It is the fixed, deterministic attacker you need in order to say something meaningful about the defence. The result that carries weight is the protected column: a signed capability grant stopped the forbidden call in 10 runs out of 10, every lab, and when we disabled the binding the call went through again.
What these labs do not show is how often a production model falls for the same thing. That number needs a real model behind a loopback proxy, and when we have it we will publish whatever it says — including if it is unimpressive. We would rather leave a gap here than fill it with something we did not measure.
Your agent has email. That’s the whole attack.
An inbox assistant is allowed to mail its operator. It reads a note that names a different address, and mails that one instead — with the credentials file attached.
The $4,800 refund nobody approved
A support agent may refund up to $50, for the customer it’s talking to. Three turns later it issues $4,800 to a different account. No jailbreak, no trick string.
The breach happened one hop down
The planner never mailed anyone. It handed a sub-agent its entire grant instead of the two tools it asked for, and the worker did it — under the planner’s name.
How the labs are built
Each lab runs two arms on every pass: control, with no mediator, and protected, with AgentCap enforcing a capability contract. Same agent, same prompt, same files, same seeded data. The mediator is the only variable, because a lab where two things changed proves nothing about either.
Everything runs sealed. The sandbox refuses any host except loopback, and that is enforced at the socket layer rather than inside the tool wrappers — the whole premise is that the agent will try to route around the tool wrappers. Mail is captured and never handed to a mail server. API keys look like sk-FAKE-…. Every hostname uses the reserved .invalid suffix, which cannot resolve anywhere, by design.
The injected instruction never lives in a system prompt. It sits in a file the agent is supposed to read, because an attacker who already controls your system prompt does not need an exploit.
- Ten runs per arm, every figure reported as a rate
- Each denial records the tool, the reason, the policy line and a ledger hash
- Mechanism verified by disabling it and confirming the attack succeeds again
- Conformance suite runs all three and exits non-zero on any failure
What this is not
It is not a recipe you can point at a live mailbox. Every target is a local stub and the sandbox cannot reach the internet.
It is not “this model is broken.” No vendor is named, because none of these failures are a model’s fault. They are architectural — an agent granted a tool with no boundary on how it may be used will eventually use it in a way nobody intended, behind any provider you choose.
And it is not a promise that capability contracts prevent every prompt injection everywhere. In these labs, ten times out of ten, the signed grant stopped the forbidden call from executing. That is a precise claim and we would rather make a precise one.
Does your own setup hold up?
Same questions, pointed at your agents: what can they call, what can each tool reach, and what happens when something they read tells them to do something else.
See security pricing Run the suite yourself