Loop engineering has emerged as a new skill for AI engineers

But there is very little research measuring how effective it is.

The best results on full tasks in a new benchmark is ~25%.

LoopArena from AMAP evaluates the outer loop rather than the coding agent.

A Controller model receives a structured summary after each round and instructs a separate fixed Worker agent on what to do or verify next, or decides to stop. Holding the Worker constant makes the result readable, since an end-to-end run cannot tell you whether success came from the guidance or from the agent carrying it out.

The named failure modes will be familiar to anyone running long agent sessions:

• Trusting a stale progress note • Skipping needed verification • Spending budget in the wrong direction • Stopping before the task is safe to submit

Paper: https://arxiv.org/abs/2608.28281

Chat with Paper: https://academy.dair.ai/papers/looparena-benchmarking-models-as-runtime-controllers-for-loop-engineering-2608.28281