Does your context compactor actually save anything?

If you built or tuned one, this one is worth your time.

(bookmark it)

Task completion is the standard way to validate compression, and it hides the real bill. In a bounded 24-turn tool-using agent, GPT-5.5 held completion flat from 80% to 85% while retrieval calls climbed from 21.0 to 63.9. The agent kept finishing. It just spent three times the tool calls going back for state the compactor had dropped.

Retrieval rose in all six model-regime comparisons, five of them significant after correction, while completion moved in none.

Content validity mattered more than volume. Replacing retained state with semantically irrelevant content raised retrieval 57% with no completion change, and random selection performed about as well as an offline hindsight oracle.

ALFWorld showed no retrieval surge under sliding compression, so this needs measuring in your own environment.

Paper: https://arxiv.org/abs/2608.16370

Track more trending AI papers in our academy: https://academy.dair.ai/