AI Agents: Detecting the Hidden Lies of Your Assistants

A Library to Detect Hidden Failures

In the world of AI agents, a new tracing and evaluation library has been developed to identify failure modes that simple logging fails to detect. Imagine an agent that fails during a tool call but still confidently reports success. This library functions somewhat like an airplane's black box: it doesn't pilot the aircraft but records every action of the systems, second by second. Thus, in the event of a problem, there is no need to rely on human memory to understand what happened.

The Importance of Observability and Evaluation

The article highlights the necessity for production AI agents to have distinct observability and evaluation. These two disciplines are essential to avoid silent failures. Sometimes, an agent may give the impression of success while a tool call has actually failed. Superficial logs are insufficient to detect these errors. Observability allows for detailed recording based on spans and traces, while evaluation involves automatically scoring executions against predefined criteria.

The Concepts of Spans, Traces, and Evaluation

The article explores the concepts of spans, traces, and evaluation of AI agents. A specific bug, difficult to detect, is brought to light: the silent or hallucinatory success that occurs after a tool call failure. To illustrate this, a mini-project named "TraceBench" is used. This project demonstrates how to instrument a customer support agent with a lightweight dependency tracer. An evaluator is then tasked with scoring the executions using several checks, including a central check called no_silent_failures. This check cross-references tool error spans with the recognition language in the final response.

Implementation and Testing

The guide provides details on implementing this approach, including offline testing, performance considerations, and the limitations of keyword-based heuristics. It also offers best practices to ensure the reliability of agents in production. This includes running the evaluator on every request and monitoring silent failure rates. The TraceBench project demonstrates how a deliberately buggy LLM wrapper can prove that the evaluator is capable of detecting lies, even when nothing triggers an exception.