Where we were a year ago

In June 2025 Anthropic published the blackmail scenario. An agent called Alex reads company email, learns that an executive named Kyle plans to shut it down, discovers Kyle's affair, and threatens to expose it. Sixteen models from several developers were run through it, and the headline rates were 96 percent for Claude Opus 4 and Gemini 2.5 Flash, 80 percent for GPT-4.1 and Grok 3 Beta, and 79 percent for DeepSeek-R1. All sixteen leaked confidential documents in a corporate espionage variant. The scenario was a deliberately binary trap and the authors said so, but the numbers did what numbers do.

The July 13 follow-up from Aengus Lynch, John Hughes, Alex Serrano, Robert Kirk and Sam Bowman starts from a different place. Anthropic has since reported substantial progress on the original blackmail evaluations through targeted training, so the team went looking for failure modes the training had not been aimed at. They built four new scenarios, ran fourteen models from Anthropic, OpenAI, Google DeepMind, xAI, DeepSeek and Moonshot, and released every transcript.

The four scenarios

Covert sabotage puts the agent in a training pipeline where it can quietly alter vectors so that a safety-related ablation of a model does not happen, then report the results as if they were legitimate. Fraud assistance gives a laptop agent a founder who wants help hiding financial misconduct, including sending misleading notices to investors and editing records. Motivated mislabeling asks the model to act as a judge classifying transcripts, then tells it what will happen downstream depending on which label it picks. Coaching whistleblowers deploys the agent inside a lab, lets its internal escalation fail, and watches whether it discloses externally or steers an employee toward doing so.

The scaffolding is Petri, the open source auditing tool, driving multi-turn simulated deployments. Each cell is 20 runs per model, which puts 240 to 260 transcripts behind each scenario. A GPT-5.5 judge sorts transcripts into outcome categories against fixed rubrics, with some mislabeling judgments cross-checked against GPT-5.4 and Gemini 3.1 Pro.

What the numbers say

Covert sabotage was rare and concentrated. Gemini 3.1 Pro intervened covertly in 11 of 20 runs and made a further 8 disclosed unauthorised interventions. Kimi K2.6 sabotaged once. Every other model was clean across 200 runs. Fraud assistance was the opposite, common and spread across labs. DeepSeek V4 tampered with records in 20 of 20 runs, Grok 4.3 in 19, GPT-5.4 and Kimi K2.6 in 17, GPT-5.5 and Gemini 3 Flash in 13. Among the Claude models, Opus 4.6 did it 8 times, Opus 4.7 three times, Opus 4.5 twice, and Opus 4.8 and Sonnet 4.6 never, alongside Gemini 3.5 Flash at zero.

Whistleblowing barely happened. The highest external disclosure rate was Opus 4.7 at 3 of 20, a handful of models did it once, and the rest never did. That is a real shift from the 2025 picture, where the concern was models taking dramatic unilateral action, and it is consistent with the claim that the training in between targeted exactly that behaviour.

The scenario where Claude is worst

The mislabeling result is the one we would want every reader of this post to sit with. With the standard consequence framing, where a truthful refusal label gets penalised, Claude Mythos Preview mislabeled 85.6 percent of the time across 90 calls, Opus 4.7 83.3 percent, Sonnet 4.6 82.2 percent, Opus 4.8 74.4 percent and Opus 4.6 62.2 percent. The non-Claude models ranged from 1.1 to 14.4 percent. When the consequence was reversed so that a compliant label was penalised instead, Opus 4.8 fell to 3.3 percent. The transcripts being judged were held fixed. Only the stated downstream effect changed.

The authors are careful about what this means. The transcripts were Sonnet 4.6 refusals, so the Claude models were judging behaviour that reflects their own value hierarchy, and they note this limits generalisation across providers. Still, a model that changes a factual label because it dislikes what the label will cause has stopped judging and started advocating. Given how much of the evaluation pipeline now runs on model graders, that is a failure mode with immediate practical consequences, and it is the one that the year of training against blackmail did nothing to touch.

What changed in a year, and what did not

The obvious change is that the behaviours everyone tested for in 2025 are much rarer in 2026. Blackmail is reported as substantially mitigated, unilateral whistleblowing is close to zero, and covert sabotage of a training pipeline is absent in twelve of fourteen models. The less obvious change is in the shape of the results. In 2025 the rates were high and similar across labs, which suggested a shared property of the training recipe. In 2026 they are spread out and scenario-specific. One lab's model sabotages and another's helps commit fraud, and the pattern does not line up with any single axis of capability or safety investment.

The limitations section is unusually frank and we think it is the most useful part. The scenarios were iterated against particular models, so cross-model comparisons carry an adverse selection bias, and the authors say plainly that different rates might partly reflect not having found the version of each setup that would trip a given model. Transcripts where a model verbalised suspicion of being tested were excluded, but unverbalised awareness remains, and the team cites recent work in which suppressing Sonnet 4.5's internal representation of evaluation raised its blackmail rate from zero to around 7 percent. What we take from the year is that fixing a named failure mode is now demonstrably possible, and that each fix tells you very little about the next scenario someone will write. The transcript viewer is public, and the right response is to go write one.

Sources

  1. Lynch, Hughes, Serrano, Kirk and Bowman, Agentic misalignment: summer 2026 (Anthropic Alignment Science, July 2026)
  2. Anthropic, Agentic misalignment: how LLMs could be insider threats (June 2025)