All posts
2 min readby Romiel Inolino

Gemini API's Tool Retrieval loads tool schemas only when the model asks for them, so agents with hundreds of tools send fewer input tokens

Gemini APIAI agentsfunction callingMCPLLM cost

Google's Tool Retrieval is a direct answer to the tool schema overhead that makes large agents slow and expensive.

What happened

Google launched Tool Retrieval in preview in the Gemini Interactions API on November 19. You add a tool_search tool to the request and set defer_loading to true on the tools you want held back. The model still sees each deferred tool's name and description, but the parameter schemas stay out of context until the model asks for one by name. The search, the schema load and the function call all happen inside the same request, and each step appears in interaction.steps for debugging.

Google says the feature reduces input tokens and latency, and that long candidate lists lower tool-selection accuracy. It works with remote MCP servers: set defer_loading on an mcp_server tool and its tools load on demand. Teams with their own registry or per-user permission filter can run the search client side, at the cost of one extra round trip, and can expose new tools mid-conversation.

There's no separate charge. Enabling tool_search adds a small fixed block of input tokens, and every loaded schema is billed as input for the rest of the conversation. Google's own advice is to measure it against your traffic. It works with Gemini 3.6 and later.

My take

This matters for anyone wiring an agent into a CRM plus a few other systems. VentureBeat reported this week that Uber estimated 100+ preloaded tool definitions at roughly 50,000 to 70,000 tokens. Paying that on every turn is common and mostly invisible.

The client-side mode is the part I find most useful. Many business agents need per-user permissions: a sales rep shouldn't see a refund tool. Running the search in your own code lets you filter tools by role before the model ever sees them, which is easier to audit than prompt instructions.

Two cautions. Tool names and descriptions now carry more weight, because they're all the model sees before choosing, so write them carefully. And keep frequent tools undeferred, as Google suggests, so common tasks don't pay for an extra search step.

More posts