Same model, three agent harnesses, nearly the same scores. The bill differed by up to 3x, set mostly by the system prompt

The wrapper around your model changes the invoice far more than it changes the result.

What happened

Two researchers at Shandong University, Yangze Liu and Zhongyi Han, published a paper titled "What Does a Harness Buy? Tokens, Mostly." A harness is everything around the model: the system prompt, the tools and the context management. They ran five models through three production harnesses (Claude Code, OpenCode and mini-SWE-agent, which gives the model only a Bash tool) on SWE-bench Verified, and reran setups to measure plain noise.

The findings, as reported by The Decoder and VentureBeat:

• On a pool of 447 tasks, Claude Code and mini-SWE-agent, the heaviest and lightest harness, landed within two points of each other.
• On the 45 hardest tasks, swapping the harness flipped about 13% of outcomes. Simply rerunning the same setup also flipped about 13%.
• The one clear harness effect was a loss: OpenCode trailed by up to 9 points, partly because of an output limit that cut steps short.
• With the same model, tasks and price list, cost per task differed by up to 3x.

The cost gap came from the preamble. The system prompt and tool descriptions, resent with every step, ran about 16,600 tokens for Claude Code, roughly 7,000 for OpenCode and under 1,000 for mini-SWE-agent. The step count then multiplies that baseline. VentureBeat also cited its own survey: 23 of 107 enterprises had no automated way to stop agent token spending.

My take

This study is about coding agents, but the math applies to every AI agent node I build in n8n, Make or Zapier. Whatever sits in the system prompt and the attached tool list gets paid for on every loop, not once.

What I take from it for client automations:

1. Measure the first call. Log the token count of the system prompt plus tool schemas before you look at anything else.
2. Attach only the tools the job needs. Twenty tools "just in case" means twenty schemas billed on every step.
3. Track step count. A lean prompt that takes twice the steps can end up costing the same, as the study saw with one model.
4. Run your baseline twice before you believe an improvement. A single run can mislead you.
5. Put a hard spend cap outside the agent so a runaway loop stops on its own.

A bigger framework rarely buys accuracy. It often buys tokens.