Agent Evaluation

Evaluating a chatbot is hard. Evaluating an agent is harder, because an agent doesn't produce an answer — it produces a trajectory: a sequence of decisions, tool calls, observations, and recoveries. A run can reach the right answer through a broken path, or fail on step 9 of 10 after doing everything else perfectly. Standard LLM evals grade the final string; agent evals have to grade the journey.

Outcome vs Trajectory

Two complementary questions, and you need both:

• Outcome eval — did the agent achieve the goal? Did the file get written, the ticket get closed, the correct number get returned? This is what users care about. Often checkable programmatically (did the test suite pass, does the DB row exist) rather than by judging text.
• Trajectory eval — how did it get there? Did it call the right tools, in a sensible order, without wasted or destructive steps? Two agents with the same success rate can differ 5x in cost and risk based on their paths.

A robust agent eval reports outcome as the headline metric and trajectory metrics to explain why the outcome happened.

What to Measure

• Task success rate — the percentage of tasks fully completed. The number that matters most.
• Tool-call correctness — right tool, right arguments, called when needed and not when not. Hallucinated tool calls and wrong arguments are a top failure mode.
• Steps to completion / efficiency — number of actions and tokens used. Loops and redundant calls show up here before they show up in cost reports.
• Recovery rate — when a tool errors or returns junk, does the agent adapt or spiral? This separates demo-quality agents from production ones.
• Cost and latency per task — agents fan out into many model calls; per-task economics, not per-call, is the unit that matters.

How to Score

• Programmatic / ground-truth checks — the gold standard when available. Run the agent against a sandbox and assert on real end state (file contents, API side effects, a passing test). No judgment needed, fully reproducible.
• LLM-as-judge — for open-ended outcomes ("was this a good research summary?"), a strong model grades against a rubric. Cheap and scalable, but biased and noisy — calibrate it against human labels and watch for it rewarding verbosity.
• Trajectory matching — compare the action sequence against a reference trajectory, or check invariants ("never deleted a file," "always validated before writing").
• Human review — irreplaceable for spot-checking and for building the golden set, too slow to be your main loop.

Building the Eval Set

• Curate from real failures. Every production incident becomes a regression case. This is the highest-value eval data you have.
• Cover the unhappy paths. Tool timeouts, empty results, contradictory inputs, ambiguous goals, malicious instructions buried in tool output. Agents fail in the messy cases, so test the messy cases.
• Use sandboxes, not production. Agents take actions. Evaluate against disposable environments (a fresh repo, a seeded test DB, mocked APIs) so a bad run can't do real damage and so state resets cleanly between runs.
• Version the eval set. As your agent's scope grows, so should the suite. Track which version of the suite a score came from.

What to Watch For

Beware the non-determinism trap: the same agent on the same task can pass or fail run-to-run, so a single pass proves nothing. Run each task multiple times and report pass rate (pass^k / average), not a one-shot result. And don't let a high outcome score lull you — an agent that succeeds 95% of the time but occasionally takes a destructive, irreversible action is far more dangerous than the headline number suggests. Weight failures by blast radius, not just frequency.