New Google DeepMind paper shows, a transformer's shallow layers never see what its deeper layers have already worked out about the context. Leaking a little of it back down during inference recovers part of what was lost.

So if your model keeps losing state over a long input, the fix may not need retraining, though the cost moves to prefill, which can no longer run in parallel.

The paper shows this working on frozen weights, but the size of the benefit does not transfer across model families.

Recirculation mixes a small fraction of a deep layer's activations into a shallow layer at the next input step, weights untouched, and sets that against the off-the-shelf model.

On Gemma3, perplexity drops on 9 of 10 language-modeling datasets, by as much as 35% for the 12B model, though multiple-choice gains are modest.

– arxiv. org/abs/2608.17981

Title: "Recirculation"