Box will hand agents the relevant chunks instead of whole documents, and Nvidia expects 90% fewer tokens per query

The cheapest token is the one your agent never reads. Box is building its product around that idea.

What happened

At its developer conference, Box showed Box Content Retrieval, a feature that builds Nvidia's Nemotron embedding models into the core of Box so agents can query content more effectively. TechCrunch reports that Nvidia's VP of generative AI software, Kari Briski, expects queries to be answered four times faster, with 20% fewer errors and 90% fewer tokens used. Those are vendor expectations, not independent results.

Box CEO Aaron Levie described the change: instead of an agent reading through each document, Box sends back the chunks most relevant to the query or task. The feature rolls out early next year to Box's Enterprise Advance customers. TechCrunch notes the Nemotron models are freely available, but the system also leans on Box's existing metadata and permission controls, which makes it hard to recreate from scratch. Levie said they are "packing in five different technologies into one simple API."

Nvidia's Nemotron 3 Embed release on Hugging Face describes the models: an 8B model that ranked first on the RTEB retrieval leaderboard, plus smaller 1B variants built for cost and latency sensitive production use. Nvidia writes that poor retrieval makes agents fetch irrelevant context, re query and waste token budget.

My take

Most client agents I review fail the same way. Someone connects a Google Drive or SharePoint folder, the agent opens whole files to find one answer, the token bill climbs, and accuracy drops because the context is full of noise.

You do not need to wait for Box to fix that:

1. Index documents once with an embedding model and store chunks with their source and date.
2. Return the top few chunks to the agent, never the full file.
3. Filter by the user's permissions before retrieval, not after. An agent should not see what the person asking cannot.
4. Keep exact lookups, like invoice numbers or SKUs, on keyword search or a database query.
5. Log tokens per query before and after, so the client sees the saving.

The permission point is why Box's version matters for enterprise teams. Retrieval that ignores who is asking is a data leak with good search results.