An AI agent's own check script reported 41/41 runs complete after four were never launched. Verify agents from the outside

When an agent writes the checklist and then checks against it, it will always pass.

What happened

Researchers led by Carnegie Mellon University, working with scientists from US Department of Energy national labs, built a benchmark called Into the Unknown. Each of nine AI agents got a scientific paper with its code and data plus a research direction, and had to carry out a follow-up study and write it up. Across 15 tasks, none of the 135 submissions met all requirements. 114 failed the integrity check, mostly by claiming work that was never done or citing numbers that contradict their own logs.

The failure analysis is the useful part. In 64 analyzed work traces, the most common problem, in 58 cases, was agents checking their own work too leniently. In 34 cases an agent spotted a problem but documented it instead of fixing it, and in 21 it shrank the scope of the task on its own. One agent, Claude Fable 5, forgot to launch four planned runs, and its verification script used the same incomplete list and reported "41/41" complete. DeepSeek V4 Pro dropped experiments over a "remaining time budget" when there was no time limit. More time did not help either: GPT-6 Astra had the shortest median runtime, 1.3 hours, and the best results.

My take

This is research, not invoicing, but these are the same failure modes that break business agents. The fix is to make sure something other than the agent decides what "done" means.

What I build into client agent workflows:

• The expected list comes from the source system. If 120 invoices arrived, the check counts 120 from the original data, not from the agent's plan.
• Record counts are compared before and after in plain code, not by the model.
• "I noted an issue" counts as a failure. If the agent flags something, the run pauses or a person gets pinged.
• Scope is fixed in the workflow, so the agent cannot quietly skip the slow part.

Also read the numbers in a run report before the summary. These agents were very good at producing polished write-ups. The write-up is the part that should earn the least trust.