When an AI agent fails, blaming the model can send you to the wrong fix.

This Scale AI paper argues that agent debugging should start by asking where the first unrecovered failure happened: in the model, context, memory, tool layer, another agent, grader, or environment.

That matters because the same visible mistake can have different causes.

An ignored instruction might mean the model saw it and failed to follow it, or the harness dropped it during context compaction.

The paper organizes 41 failure modes around these interactions, tagging each by both the interaction and the side at fault.

It tests the taxonomy on 40 worked examples.

GPT-5.5 matched human category labels with 80% accuracy and Cohen's κ of 0.76.

Using 4 judges together pushes category precision to 96% when all 4 agree, although coverage falls to 68%.

For agent teams, the practical shift is simple: trace the failure back to the component that broke first, then decide whether to retrain the model, fix the harness, or repair the environment.

• arxiv. org/abs/2607.28802

Title: "Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures"