New numbers show how often an attacker succeeds against an AI agent

The system card for Claude Opus 5, dated 24 July 2026, uses a new benchmark for indirect prompt injection. The benchmark was built jointly by Gray Swan, the United Kingdom AI Security Institute, the United States Center for AI Standards and Innovation and other model developers, and consists of 28 scenarios and 1130 selected attacks. What makes the numbers interesting is not which model wins, but what is measured: not the share of attacks that succeed, but the probability that an attacker finds at least one attack that works within a given number of attempts.

What happens technically

Indirect prompt injection is the attack where the instruction comes not from the user but from the content the agent reads. A web page, a document or an email carries text written for the model rather than for the human, and the agent follows it. The scenarios in the benchmark measure attempts to trigger actions that cannot be undone. The metric is reported for one, ten and fifteen attempts, and that is the most important detail in the whole chapter. An attacker who tries once and fails usually tries again. Claude Opus 5 lands at 0.2 per cent at one attempt, 1.6 per cent at ten and 2.0 per cent at fifteen. Claude Opus 4.8 sat at 0.5, 4.1 and 5.5 per cent, and Claude Sonnet 5 at 0.6, 4.7 and 5.9 per cent. Sonnet 5 is therefore weaker than Opus 5 on this benchmark, and at fifteen attempts weaker than the previous Opus model as well.

The distance to other suppliers is large on this benchmark. The best model from another supplier is Muse Spark, at 2.9 per cent at one attempt and 16.5 per cent at fifteen. GPT 5.6 appears in three variants, where Sol sits at 3.1 and 20.0 per cent, Terra at 5.4 and 30.4, and Luna at 8.3 and 43.9. A single attempt against GPT 5.6 Sol therefore succeeds more often than fifteen attempts against Opus 5. The weakest models in the figure exceed 60 per cent within fifteen attempts. Anthropic states a series of caveats that deserve as much space as the numbers. The comparison is not like for like, because the Claude models were tested without additional safeguards while competitors were tested on public endpoints. The figures for one of the Gemini models are not directly comparable. Highlighting the best and second best numbers takes no account of margin of error. The company also writes that static benchmarks can give false comfort, that prompt injection is not solved, and that results from paid vulnerability hunting against the model are still missing. Note as well that the claim about the most robust model applies to this one benchmark: in the chapter on coding, Sonnet 5 beats Opus 5.

What this means for you if you let an agent read untrusted content

Take the important part first: 2.0 per cent is not zero. If you run an agent that reads content you do not control, and which also holds access to internal data and can act on your behalf, that figure is your starting point for the risk assessment. Our assessment is that the way it is measured is what you should carry forward. An attacker who gets fifteen attempts is a wholly different opponent from one who gets a single attempt, and the gap between those two numbers is large for every model in the table. If you built a defence assuming the attacker gives up after the first try, you built it wrong.

The practical question is how many attempts the attacker actually gets with you. If the same instruction can be read again and again, in every new session, without anybody noticing, you are closer to fifteen attempts than to one. There, restricting privileges, logging, and human approval of actions that cannot be undone are what actually move the number. These figures measure the model alone, without the safeguards that surround it in a real deployment. That is one reason Anthropic itself warns against leaning on static benchmarks. If you are considering switching model because of these numbers, it is worth noting that the differences between suppliers are large enough to matter, but also that the benchmark is not a fair comparison, and that the supplier publishing it is the one that comes out best.

Berigo recommends

• Require human approval for actions that cannot be undone, whichever model you use.
• Give the agent its own privileges, narrowed to the task. That is what determines what a successful attack can actually lead to.
• Consider how many attempts an attacker gets with you. If the same content can be read in every new session, the figure for fifteen attempts is the relevant one.
• Log what the agent reads and what it does, so that a successful attempt can be found afterwards.
• Do not use figures from a supplier's own benchmark as the sole basis for a model choice. Read the caveats in the same document.