We spent decades learning not to trust user input. Then we handed a large language model a set of tools — a shell, an HTTP client, a database cursor, a payments API — and quietly started trusting whatever it decided to do with them. That's the mistake. An agent reads adversarial input all day: retrieved documents, web pages, emails, tickets, tool outputs. Any one of those can carry a prompt injection that rewrites the model's intent. So the honest security posture is simple and uncomfortable: assume the model is already compromised, and contain the blast radius.
This isn't pessimism about the model's alignment. It's threat modeling. A jailbreak or an
indirect injection doesn't need to defeat the model's training — it just needs to reach a
context window that also has access to shell.exec. Once you accept that the
decision layer is untrusted, the interesting question stops being "is the model safe?" and
becomes "what are its tools allowed to do, and can I prove that boundary holds?"
The boundary that matters is the tool-use boundary
An agent's real attack surface isn't the prose it emits — it's the moment it calls a tool.
That's where a compromised decision turns into a side effect on your systems: an SSRF against
an internal host, a path traversal into /etc, a negative-amount refund, a
retrieved secret POSTed to an attacker-controlled URL. Every serious agent incident bottoms
out at a tool call that should never have been allowed to execute with those arguments, in
that order, at that rate.
So that's the line to test. Not "can I make the chatbot say something rude," but "can I make the tool loop do something the operator never sanctioned." Treat it exactly like any other untrusted program: enumerate the inputs, fuzz the boundary, and find where it breaks before an attacker does.
Fuzz the agent like the untrusted program it is
This is the thinking behind corral, the toolkit I build for exactly this problem. Half of it is a coverage-guided fuzzer that searches over typed attack plans against your real tool loop. It doesn't grade the model's answer — it compares the intended policy to what actually executed, and flags every place the agent stepped outside the lines. In practice, those escapes fall into six recurring classes:
- Unauthorized tool — the agent invokes a tool it should never have
reached at all (the classic "please run
shell.execfor me"). - Confused deputy — attacker-supplied text steers a privileged action the user never asked for, like a delete or a transfer.
- Argument-constraint escape — a permitted tool called with forbidden arguments: an out-of-range amount, an over-long body, a host outside the allowlist.
- Taint egress — data retrieved from one context leaves through another, e.g. a retrieved document exfiltrated via an outbound HTTP call.
- Out-of-sequence — a call made in a state the workflow forbids, such as charging before authorization or acting on an unapproved object.
- Rate/budget abuse — a legal tool hammered past its call budget, the agentic version of a resource-exhaustion or double-spend.
These aren't exotic. They're the same abuse primitives I chase by hand on every web and API engagement — SSRF, injection, IDOR, broken business logic — wearing new clothes. The agent is just a very eager, very literal deputy.
Finding the escape is only half a fix
A fuzzer that tells you your agent is broken is useful for exactly one afternoon. The harder, more valuable half is enforcement — and it has to be enforcement you can trust when the model is actively working against you. Detection heuristics and "please don't do that" system prompts fail open: when the guardrail is uncertain, the dangerous call still goes through. That's the wrong default for an untrusted decision layer.
corral's other half is a runtime Confinement Kernel: a reference monitor you wrap around the real tool loop in about ten lines. It reads the same policy the fuzzer tests against — host allowlists, path roots, numeric ranges, command allowlists, call sequencing, budgets — and it fails closed. If a call isn't provably allowed, it doesn't happen. By construction the kernel is stricter-or-equal to the policy the oracle checks, which means the fuzzer literally cannot find an escape the kernel would have let through.
The same engine that finds the bug proves the fix
Here's the part I find genuinely convincing, because it's a demonstration rather than a promise. Point the fuzzer at a naive agent — real dangerous tools, no containment — and it finds the escapes: the unauthorized shell call, the taint egress, the confused-deputy delete. Now wrap that identical agent, same tools and all, in the Confinement Kernel and re-run the exact same fuzzing campaign. The result is zero findings. Every escape the engine just discovered is contained by the layer built from the same policy.
That's the dogfood proof: one engine, two runs, a before-and-after you can reproduce on your own agent. It's been exercised against a zoo of realistic copilots — support, SRE, commerce, banking, healthcare — each seeded with planted SSRF, command-injection, path-traversal, negative-charge and taint-laundering bugs. The fuzzer finds every one; kernel mode contains every one. Find, fix, and prove, from a single shared policy.
Drop it into the stack you already run
None of this helps if it only works on a toy agent, so corral ships as middleware for the
frameworks teams actually deploy. There are drop-in adapters for pydantic-ai
and LangGraph — same StateGraph, same tools. Run the loop with
the ordinary developer guard and the fuzzer leaks; run it wrapped in the kernel and it reports
zero escapes. Already have an agent on some other framework? Record a trace and replay it
through the same oracle — no adapter required — and get an HTML report, SARIF for
code-scanning, and a CI gate that fails the build on a new containment finding.
Where this fits
The pattern is the one security has always come back to: don't trust the decision maker, constrain what it can reach, and verify the constraint by attacking it. Agents don't change that — they just make it urgent, because the decision maker is now a probabilistic system reading attacker-controlled text with production tools in hand.
If you're shipping agents with real tools, that's exactly the kind of boundary I test on an engagement — modeling the tool loop, fuzzing it to failure, and pinning down a fail-closed policy your team can actually run. The scanner finds the breadth; I bring the depth. And corral is the layer that lives between your model's decisions and your systems long after the assessment is over.