Interesting technical work from Microsoft.

Provides a better understanding on SFT and how to leverage it better for RL.

Microsoft researchers asked whether a standard SFT pipeline actually produces the model you want to run RL on.

Their answer is no.

Standard SFT keeps spending gradient on sequences the model has already fit, which narrows the distribution RL later needs to explore.

TailSFT filters those sequences out during training and concentrates learning on the under-modeled tail of the data.

That is the only modification they implement.

Results:

On OLMo-3 7B, pass@16 improves by up to 16.8 points absolute on coding and 3.1 on math. Those higher-coverage checkpoints then lift final pass@1 after GRPO by up to 3.9 points, and in some settings early reward climbs 2.5x faster than the matched standard SFT run.

Paper: https://arxiv.org/abs/2608.25756

Chat with Paper: https://academy.dair.ai/papers/tailsft-filtered-fine-tuning-improves-post-training-performance-2608.25756