# TailSFT：微软提出过滤式SFT提升RL性能

- 来源：DAIR.AI (@dair_ai)
- 发布时间：2026-08-31 03:00
- AIHOT 分数：44
- AIHOT 链接：https://aihot.virxact.com/items/cmtg7xbot05geroj7e1uxakpa
- 原文链接：https://x.com/dair_ai/status/2094138107314753938

## AI 摘要

微软研究指出标准SFT会持续在模型已拟合的序列上消耗梯度，收窄了后续RL所需的探索分布。TailSFT在训练中过滤这些序列，专注学习数据中欠拟合的尾部。在OLMo-3 7B上，pass@16在编程任务上最高提升16.8个绝对点，数学提升3.1点；经GRPO后最终pass@1最高提升3.9点，部分场景早期奖励提升速度快2.5倍。

## 正文

Interesting technical work from Microsoft.

Provides a better understanding on SFT and how to leverage it better for RL.

Microsoft researchers asked whether a standard SFT pipeline actually produces the model you want to run RL on.

Their answer is no.

Standard SFT keeps spending gradient on sequences the model has already fit, which narrows the distribution RL later needs to explore.

TailSFT filters those sequences out during training and concentrates learning on the under-modeled tail of the data.

That is the only modification they implement.

Results:

On OLMo-3 7B, pass@16 improves by up to 16.8 points absolute on coding and 3.1 on math. Those higher-coverage checkpoints then lift final pass@1 after GRPO by up to 3.9 points, and in some settings early reward climbs 2.5x faster than the matched standard SFT run.

Paper: https://arxiv.org/abs/2608.25756

Chat with Paper: https://academy.dair.ai/papers/tailsft-filtered-fine-tuning-improves-post-training-performance-2608.25756