Insightful paper from Microsoft and colleagues.

If you have ever had an agent run fail 80 steps ago with no way to find where, this one is for you.

(bookmark it)

Agent failures show up as long complex trajectories. Traditional software debugging techniques do not apply here, and handing the whole trace to an LLM judge produces unreliable diagnoses.

AgentScope is a neuro-symbolic diagnosis system addressing this issue. Agent behavior is abstracted from its trajectory into a structured representation, so the search happens over program-like objects instead of prose.

Behavior properties are then written as neural invariants, specifications stated in natural language that an LLM checks against the abstraction.

That combination identifies both the failing step and its failure type. It significantly outperforms the current state of the art in fault localization and attribution accuracy on the public Who&When dataset and on AgentErrata, a broader failure dataset the authors built.

Paper: https://arxiv.org/abs/2609.02371

Chat with Paper: https://academy.dair.ai/papers/diagnosing-with-insights-structured-analysis-of-agent-failures-via-behavioral-ab-2609.02371