All posts
2 min readby Romiel Inolino

LangChain raised $300M at a $3B valuation. Its own guide to building agents says: write the eval before the change

LangChainAI agentsevalsLLMOpsautomation

The money is going to companies that help teams know whether their agent got better or worse. That tells you where the real pain is.

What happened

LangChain raised a $300M Series C at a $3B valuation, led by CapitalG with GIC, DFJ Growth, IVP and Sapphire Ventures. TechCrunch notes that is more than double its $1.25B valuation from about a year ago. LangChain says more than 7,000 organizations use LangSmith, its platform to run, improve and govern agents, and cites Workday cutting the time to deploy a new agent from five months to under a day.

The same week, LangChain published a guide to eval-driven development. The method has three steps: define what good looks like, write the eval before you build, then make the change and compare experiments.

Its worked example is a docs agent that gets a new create_ticket tool. The spec lists when to file a ticket, when not to (how to questions), and what must keep working. After adding the tool, the ticket cases passed and the average score went up. But "How do I connect to the VPN?" now filed a ticket instead of searching the docs. The trace showed the tool description, "VPN and laptop issues", was too broad. Narrowing it fixed every case.

The guide also warns against tuning until a small set hits 100%, recommends a held out set to catch overfitting, and lists common mistakes: loose success criteria, writing the eval after the fix, and only checking the average.

My take

That VPN example is the most common failure I see in client automations, with or without LangSmith. Someone adds a capability, the new thing works, and an old path breaks silently.

You do not need a big platform to start. For any agent that touches a CRM or a client inbox:

  1. Keep 20 to 30 real inputs in a sheet, each with the action it should trigger.
  2. Include the ones that should NOT trigger the new action.
  3. Run them before and after every prompt or tool change, and compare row by row, not just the total.
  4. Turn every production mistake into a new row.

Agents are non deterministic, so a test list is the only honest way to say a change helped.

More posts