Gemini 4 Carbon tops MathArena at about $15.70 per run versus $1.07 for GPT-6.1 Sol, and it once ran a script for 11 hours unchecked
Google's Gemini 4 Carbon now leads the MathArena math benchmark, and the failure notes are as useful as the score.
What happened
MathArena added results for Gemini 4 Carbon and Claude Fable 5.2 on November 13. The Decoder reports that Carbon took the top spot overall with an expected 85.4%, ahead of Claude Fable 5.2 at max settings with 82.8% and GPT-6.1 Sol at max with 80.6%. The best open model listed, Kimi K3.1 Pro, came in at 54.3%.
Cost separates them sharply. With the reasoning settings used in testing, the expected cost was about $15.70 for Carbon, $40.20 for Fable 5.2 and $1.07 for GPT-6.1 Sol, which The Decoder describes as roughly 15 and 40 times the cost of GPT-6.1 Sol.
The behavior notes come from MathArena's Jasper Dekoninck, as reported by The Decoder. Carbon's most common failure mode is exceeding the time limit, and it almost never gives up on a task. In one case it launched a script that ran for more than eleven hours without checking on its progress.
My take
A math benchmark doesn't tell you much about CRM updates or lead routing, so I wouldn't pick a model for business automation from this ranking. The cost spread and the failure pattern are what carry over.
A model that keeps going until it hits the time limit is fine in a benchmark that enforces one. In a client's agent workflow, the limit has to come from you. If an agent can start a long-running script or keep calling tools, I set a wall clock timeout per step, a cap on tool calls per run, and a spending limit per task. When a limit trips, the run stops, records where it was, and alerts someone, instead of retrying quietly.
The 15x price gap is also a reminder to price per finished task. A cheaper model that scores a few points lower may be the better choice for routine steps, with the expensive one kept for the cases where the cheap one fails your evals.
These figures come from one benchmark and The Decoder's summary of it. I couldn't read the full per-task breakdown, so I'd treat the exact costs as indicative.
More posts
- Open OCR models now read 1,000 pages for under $0.10, and better serving saved more than switching modelsNov 20, 2026
- Gemini 4 Neon costs $3 in and $15 out per million tokens and tops Zapier's AutomationBench at 56.3%, though it uses more output tokens per taskNov 20, 2026
- Gemini API's Tool Retrieval loads tool schemas only when the model asks for them, so agents with hundreds of tools send fewer input tokensNov 20, 2026
