New Microsoft + Cornell Univ paper gives a method that matched a standard model trained on 50% more tokens, while adding only 14% more training time and ~1% inference latency.

Normally, the same hidden state has to do 2 jobs: keep track of the context and predict what comes next.

The paper gives prediction its own extra computation, but without adding another token, growing the KV cache, or adding another decode step.

You do not need it for the whole training run.

When the free pause was switched on after 42.5% of training, it kept about 94% of the full quality gain while training took 1.33× the wall-clock time of the normal model.

At equal node-hours, the phased versions still beat standard training.

So the recommendation is: train normally for most of the run, then add the extra prediction computation near the end.

– arxiv. org/abs/2609.03807

Title: "Free Pause Tokens"