All posts
2 min readby Romiel Inolino

Your MCP agent can state a true fact and still cite the wrong source. ProvenanceGuard checks for that

MCPAI agentsverificationRAGAI engineering

"Grounded" can mean supported by any tool output, or supported by the source the answer actually names. Those are different tests.

What happened

Researchers at Multiverse Computing published ProvenanceGuard, a verification layer for agents that use the Model Context Protocol. It targets what they call cross-source conflation: a claim that is true somewhere in the evidence but attributed to the wrong source. Their example is a support agent saying "according to the account record, this plan includes a 30-day refund window" when that fact actually lives in a policy document.

ProvenanceGuard runs after the agent answers. It reads the captured MCP trace with tool and source IDs, splits the answer into claims, finds the most relevant source for each, checks support, compares it with the source the answer names or implies, and returns per-claim verdicts plus an allow or block decision. Numbers, dates and identifiers missing from the source cannot pass just because the sentence sounds plausible.

On 281 medical-domain agent traces, experts said 139 held-out claims should not pass, and ProvenanceGuard caught 138. It also held 67 claims experts considered supported, sending them for review. It detected all 50 deliberately swapped attributions. On a harder test with several similar sources, it identified the exact source for only 50.3 percent of claims, which the authors flag as an open problem. They report roughly half a second of overhead per answer in their local setup.

My take

This matters well outside medicine. Any CRM or support agent that pulls from a contact record, a policy doc and a knowledge base can mix them up. A sales rep who reads "per the account record" trusts it more, which makes a wrong attribution worse than a vague answer.

Practical steps for agents you run today:

  1. Log every tool call with a stable source ID. Without the trace, you cannot verify anything later.
  2. Make the agent name its source per claim when it matters, such as pricing, refunds and contract terms.
  3. Check literal values. A number or date in the answer should appear in the named source. That check is cheap and catches a lot.
  4. Accept some over-blocking on sensitive answers. A fallback like "let me confirm with the team" beats a confident wrong citation.

Source checks like these are standard in what I ship: support agents, CRM data entry, reporting and research pipelines that cite where every answer came from. Browse the range at romielwillautomate.dev.

More posts