New Stanford paper Prefix Sliding shows that long reasoning does not need the full chain of thought in memory: keep the task prefix and recent tokens, and inference can run about 3x faster without retraining.

Long reasoning gets expensive because full attention makes every new token look back over an ever-growing chain of thought.

Prefix Sliding keeps the fixed prefix with task and tool instructions plus a sliding window of recent reasoning, dropping older middle tokens as it goes.

Their attention analysis points in the same direction: the prefix and latest tokens receive most attention, while intermediate reasoning gets little.

On Qwen3-1.7B, a 4,096-token window scored 33.9% on AIME25 versus 34.2% with full attention, and the paper reports about 3x faster inference without retraining.

Once the window fills, each new token has constant attention cost instead of becoming more expensive with every step, which also enabled reinforcement-learning rollouts beyond 100,000 tokens.

Prefix Sliding beat pure sliding windows, repeated summarization, and last-k deletion on the tested speed/accuracy tradeoff.

– arxiv. org/abs/2608.26070

Title: "Prefix Sliding for efficient test-time scaling"