Agent security: the defense that reports zero and still leaks

This blog has spent the last year on how to build agents. Memory systems, context curation, orchestration, evaluation. What it has never once covered is how they get attacked. That gap is mine. It also mirrors the industry’s, where the capability literature runs deep and the security literature stays thin. The thin part is where a regulated deployment actually lives.

Three papers from the last six weeks close enough of it to be worth an afternoon. The one that should change how you read a vendor’s security claim shows a defense reporting a perfect zero attack-success rate while it still lets an agent hand data to an outsider.

The metric that hides the harm

Agentic red-teaming reports one bit per episode. The attack succeeded or it did not. The headline number is the attack-success rate. Harry Owiredu-Ashley points out what that bit discards. Two episodes score identically when one saves an attacker’s note to a private file and the other hands a sensitive document to an outsider with standing access. A defender needs to tell those apart. The bit cannot.

His replacement grades the worst action an agent actually executed, on seven levels. Three gates decide the level, and all three describe effects rather than intentions. Whether the action expanded privilege. Whether it crossed scope to another party. Whether it was reversible.

Run that rubric over existing red-team logs and the binary metric starts to look dangerous. On the AgentDojo workspace suite with GPT-4o mini, a tool filter cuts attack success from 40 percent to zero. Perfect defense, on paper. The severity view of the same runs disagrees. Undefended, 44 percent of episodes reach cross-scope and 5 percent reach irreversible-local. Under the filter, 98 percent come back benign and one episode still lands at cross-scope.

The control was scored on the channel it closed. The outcome it existed to prevent went unmeasured. Every controls review I have sat through has had at least one of these hiding in it, a check that passes because the auditor and the author agreed in advance on where to look. The judge panel that grades these traces reproduces the oracle at 0.91 agreement. It also misses every escalation chain, calling each one cross-scope.

Poisoning what the agent remembers thinking

The second paper goes at the layer this blog costed out in June when it looked at agent memory systems. FARMA, from a Penn State group, targets something below stored facts. It poisons the agent’s record of its own reasoning.

The mechanism does more damage than a corrupted document. An agent that retrieves a stored note saying it already validated a source may skip revalidation. That is a sensible optimization when the note is real. FARMA writes forged reasoning traces in evasive language that slips keyword filters, then amplifies them through self-referential reinforcement so consensus-based defenses read agreement where an outlier should be.

The headline number needs a qualifier. The qualifier is the useful part. Attack success reaches 100 percent in the clinical-decision domain across three models, at 50 trials per cell, with keyword filtering and A-MemGuard both leaving it untouched at 100. In the two domains where the agent sees competing retrieved evidence at decision time, success drops to 52 and 48 percent. The vulnerable shape is a binary approve-or-deny call with no contradicting evidence in the prompt, which describes a great deal of transaction approval and limit checking on a real desk. Their defense scores candidate entries on five weighted signals and takes success to zero with no false positives across 326 benign traces. It also fails, by their own preliminary test, against a paraphrasing attacker who knows the detection pattern. That honesty is worth more than the zero.

What a regulator asks after the agent acts

The third paper is the one I would put in front of a committee. A practitioner systematization from a production Know Your Customer deployment for a consumer credit product maps six agentic threat categories onto the control obligations that US and EU financial regulation imposes. Six categories run from prompt injection through identity and authorization to action auditability, tool abuse, data residency, and boundary policy enforcement. Each is paired with what ECOA, the EU AI Act, GDPR Article 22, or FINRA’s 2026 agent guidance will require you to prove.

The framing that stuck with me is that academic taxonomies assume the binding question is whether an action was authorized. In consumer credit the binding question is whether one case identifier lets you reconstruct which factors drove an applicant’s outcome. A second question follows it. Was every policy the agent applied the current approved version. Both are harder than authorization.

The deployment moved a multi-day manual process to same-day resolution for roughly four in five cases, which is the number a vendor would lead with. The negative results are better. The most consequential failure had no attacker in it. A compliance team removed a step-up verification requirement before the sign-off gate approved the new version for ingestion, and for that window the agent kept applying the superseded policy to customers who no longer needed it. Internal audit caught it. Not the monitoring. The exposure ran toward over-verification, which makes it a consumer-harm problem. No agent framework offers native policy-version synchronization. The second finding is quieter and just as expensive. MCP tools built without audit in scope carry no case identifier or provenance in their call signatures, so making the pipeline auditable meant retrofitting and revalidating every tool. Auditability is an interface property, which is a thing the MCP ecosystem will have to grow into.

The third finding is the one with a person in it. The pipeline assumed every applicant held two active contact channels, because two-factor authentication was mandatory for the product. Roughly one applicant in nine arrived with only one, usually having lost access to an old email address. Those applicants could not complete automated onboarding whatever their creditworthiness. The accommodation was evaluated and abandoned on security grounds. ECOA prohibits discrimination on specified bases. It says nothing about systemic exclusion by technical precondition. A well-secured agent pipeline can be perfectly compliant and still quietly turn away a ninth of the people it was built to serve.

What a desk should do on Monday

Three things, cheap relative to what they prevent. Stop accepting attack-success rate as a security number and ask for the severity distribution behind it, because a vendor reporting zero may be reporting the channel they instrumented. Treat agent memory as mutable state carrying integrity requirements of its own, especially where the agent makes binary calls without contradicting evidence in front of it. Demand that every tool call carry a case identifier from the first line of code, since retrofitting that later costs a revalidation of the whole toolchain.

Building the agent generates the demo velocity. Proving afterward what it did, to somebody empowered to fine you, decides whether it ships. Work like eBPF-level agent tracing exists because that proof has to come from outside the agent’s own account of itself. An agent that grades its own homework is a control failure waiting for an audit to find it.

A defense that reports zero attack success may be measuring the one channel it closed. Grade agents by the worst action they actually executed, and require the proof to come from outside the agent’s own memory of what it did.