# 并非所有去噪步骤都同等重要：通过模型调度加速掩码扩散语言模型

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-11 08:00
- AIHOT 链接：https://aihot.virxact.com/items/cmnygovrl0039sl134f2l82m1
- 原文链接：https://arxiv.org/abs/2604.02340

## AI 摘要

研究人员提出模型调度策略以降低掩码扩散语言模型（MDLMs）的采样成本。该方法在特定去噪步骤用小规模模型替代完整大模型，基于早期和晚期步骤对模型替换更鲁棒的发现，在OpenWebText和LM1B数据集上实现FLOPs减少17%，仅带来生成困惑度的轻微下降，同时保持样本多样性。通过损失函数与KL散度的步骤重要性分析证实，扩散轨迹中段对模型替换最为敏感。这一架构无关的调度方法可在基本保持生成质量的前提下显著加速MDLM采样。

## 正文

Recent advances in masked diffusion language models (MDLMs) narrow the quality gap to autoregressive LMs, but their sampling remains expensive because generation requires many full-sequence denoising passes with a large Transformer and, unlike autoregressive decoding, cannot benefit from KV caching. In this work, we exploit the flexibility of the diffusion framework and study model scheduling, where a smaller MDLM replaces the full model at a subset of denoising steps. Across models trained on OpenWebText and LM1B, we show that early and late denoising steps are substantially more robust to such replacement than middle steps, enabling up to a 17% reduction in FLOPs with only modest degradation in generative perplexity under both unconditional and prefix-conditional generation, while preserving sample diversity. We support these findings with a step-importance analysis based on loss and KL divergence between small and large models across timesteps, as well as an exhaustive search over coarse step segments, both of which identify the middle of the diffusion trajectory as most sensitive consistently across datasets. Our results suggest that simple, architecture-agnostic scheduling rules can significantly accelerate MDLM sampling while largely preserving generation quality.