The best AI agent picked the better path at a key fork only 59.7% of the time. Put checkpoints where your workflows branch

Agents are good at finishing steps. Choosing which road to take early in a long job is a different skill, and it is weaker.

What happened

Researchers from Microsoft and City University of Hong Kong built Taste-Bench, a set of 502 questions taken from real agent runs on coding and AI research tasks. Each question freezes a run at a "decision fork", where two directions were possible and one later turned out better, hides everything after it, and asks a model to pick.

A question counted only if the model chose correctly with the options in both orders, so guessing scores 25%. Of 14 models tested, the best, GPT-5.6 Sol, scored 59.7%, with GPT-5.5 at 59.5%. When the deciding clue was already visible before the fork, average accuracy was 62.3%. When it only showed up after a lot of later work, accuracy fell to 21%. Raising the reasoning setting barely changed results.

The authors also showed taste can be trained: a student model distilled from a teacher that had seen outcomes made better choices on unseen tasks.

My take

Long running automations are full of forks. Which records to treat as duplicates. Whether to rebuild a pipeline or patch it. Which lead source to trust when two disagree. A bad call looks fine at first and only costs you later, which is exactly where these models were weakest.

How I handle it:

1. Make the big forks design decisions. I decide the structure of a client workflow up front; the agent fills in the steps.
2. Put a check soon after each branch, so a wrong turn shows up in minutes, not at the end of a 3 hour run.
3. Add a human checkpoint before anything hard to undo, like merges, deletes or bulk sends.
4. For high value tasks, run two cheap attempts and compare the outcomes instead of trusting one path.
5. Do not expect a bigger thinking budget to fix it. The study says it does not.

Agents can do the work. You still own the route.

Got a long, branching process you would like an agent to run? Message me through romielwillautomate.dev and describe it, and I will map where the checkpoints go.