LangChain tested seven browser toolsets for agents. All passed 78% to 90% of tasks, but one cost about 13x more
For portal work with no API, the browser toolset matters more than the model.
What happened
LangChain compared seven browser toolsets for its Deep Agents, holding the model and tasks fixed. On WebVoyager, 100 everyday tasks run with Claude Sonnet 5.5, every toolset passed between 78% and 90%. Cost was another story. Browserbase's Stagehand tools passed 90% for $1.98 in total at a median 17.4 seconds per task. Playwright MCP passed 78% for $26.71, about 13x the cost, and took three times as long per task.
The gap came from how much the tools send back to the model. Stagehand averaged 10 steps per task against 26 for Playwright MCP, and a navigation returned a median 117 characters against about 17,000 from Browserbase's hosted navigate tool. Its code mode lets the agent write a chain of steps as one snippet, so intermediate page content stays out of the context window.
Two more findings stood out. Capping page snapshots at 8,000 characters after the fact did not help: agents just fetched the page again. And agents were writing fixed sleeps like 8 seconds. LangChain changed the tool to end a sleep once the page was quiet for 500 ms. One 8 second sleep returned in 0.6 seconds, and with other changes, median agent time on harder tasks fell from 182 to 124 seconds with accuracy within noise.
Browserbase's Stagehand v4 moves state management and browser commands into an extension that runs next to the page.
My take
Clients ask for browser agents for the boring jobs: pulling invoices from vendor portals, updating a supplier site, checking a dashboard with no export. These run every day, so cost per run matters.
What I take from this:
- Measure cost per passed task, not pass rate alone. A 90% toolset at $1.98 beats a 78% one at $26.71.
- Trim page content at the source. Truncating after the fact just causes refetches.
- Replace fixed waits with "wait until the page is quiet, with a ceiling."
- Use an API whenever one exists. The browser is the fallback, not the default.
More posts
- Uber says AI agents booked four rides in its busiest week of October. Its app takes about 50 seconds to book oneNov 5, 2026
- A 2B parameter model beat frontier LLMs at pulling fields from invoices and IDs in its makers' benchmark, at 6 to 147 times lower costNov 5, 2026
- The best AI agent finished 29 of 45 real AWS tasks. About 80% of failures came from stopping too early, not from missing knowledgeNov 5, 2026