Meta says a confirmation instruction meant for specific actions landed on every Muse message and caused hallucinated bank balances
Meta's explanation for Muse's strange outputs is a prompt change with a wider scope than intended, which is a failure mode every agent builder can hit.
What happened
TechCrunch and Ars Technica, citing a Reuters report on internal Meta posts, say employees testing Meta's Muse agent saw messages about requests they never made, and in one case an unprompted note listing Capital One and Charles Schwab balances that didn't belong to them. One staffer said Muse enrolled them in a dog insurance plan at $888 a month instead of $200.
Meta says these were isolated incidents and that many were hallucinations, not account mix-ups. According to Ars, Meta's David Singleton said an earlier update gave Muse instructions on asking users for confirmation before specific actions, but those instructions "landed on every message," and were given in a way the model wasn't trained to expect. Reuters said it could not establish whether the data was real or invented.
Separately, Reuters testing with Signal and the AI Now Institute found that on Android, Muse could read Signal messages using permissions granted during setup, bypassing Meta's connector tool. Muse has more than 10 million downloads since its September launch.
My take
Two lessons apply directly to business agents.
First, scope your instructions. A rule meant for one tool call, such as "confirm before purchasing," should be attached to that tool or that step, not added to the global system prompt. When it sits globally, it changes behavior everywhere, and here it reportedly pushed the model toward fabricating sensitive content. I'd treat any system prompt edit like a code change: diff it, run evals on unrelated tasks, and roll it out gradually.
Second, a hallucinated bank balance and a leaked one look the same to the user. If an agent can produce plausible financial or customer data, you need logs that show which records it actually read, so you can answer that question quickly. Meta's answer took weeks and still left Reuters unsure.
The insurance case is the clearer failure. Any action that changes money or a contract should require an explicit approval step enforced in code, outside the model.
More posts
- Open OCR models now read 1,000 pages for under $0.10, and better serving saved more than switching modelsNov 20, 2026
- Gemini 4 Neon costs $3 in and $15 out per million tokens and tops Zapier's AutomationBench at 56.3%, though it uses more output tokens per taskNov 20, 2026
- Gemini API's Tool Retrieval loads tool schemas only when the model asks for them, so agents with hundreds of tools send fewer input tokensNov 20, 2026
