New Microsoft plus Univ of San Diego paper shows a model can look better after SFT but actually be a worse starting point for RL, because SFT may wipe out rare correct behaviors that RL needs to discover.

TailSFT preserves more of those behaviors, and with the same RL setup it produced up to 3.93 percentage points higher final pass@1.

standard SFT can make RL harder by overtraining already-fit examples; TailSFT filters them and gives the later RL stage a better starting point.

TailSFT changes SFT by filtering sequences whose loss has already dropped most relative to the base model.

That shifts training toward examples the model still underfits, with one goal: keep correct responses reachable under repeated sampling so RL has useful behavior to reinforce.