当置信度产生误导:面向扩散语言模型的后缀锚定与锚点邻域置信度调节
阅读原文· arxiv.org扩散语言模型通过迭代去噪掩码token序列解码文本,置信度常被用于选择解码位置。然而,高置信度有时会产生误导,例如EOT token可能获得高置信度导致生成不完整。为缓解此问题,插入后缀锚定可鼓励生成完整响应,但会引入锚点邻域的局部过度自信,导致锚点邻近token过早解码。为此,研究提出了后缀锚定置信度调节方法,该方法插入短后缀锚点以促生成完整响应,并根据解码进度调节锚点附近置信度。在纯文本推理、视觉-语言推理和代码生成基准测试中,该方法持续提升了基于置信度的完全非自回归解码性能,优于显式EOT抑制,并保留了完全非自回归生成的并行优势。
Diffusion language models decode text by iteratively denoising masked token sequences, making the choice of which positions to decode a central inference-time decision. Most training-free decoding strategies use model confidence for position selection, assuming that high-confidence positions are ready to be decoded. In this work, we revisit this assumption by studying when confidence misleads fully non-autoregressive (fully non-AR) decoding. EOT tokens can receive high confidence and cause incomplete generation; inserting a suffix anchor can mitigate this issue but introduces local overconfidence near the anchor, causing anchor-adjacent tokens to be decoded too early. To address these issues, we propose Suffix-Anchored Confidence Modulation, a simple training-free method that inserts a short suffix anchor to encourage response completion and modulates confidence near the anchor according to decoding progress. This preserves the response-completion benefit of suffix anchoring while reducing premature decoding of anchor-adjacent tokens. Across text-only reasoning, vision-language reasoning, and code-generation benchmarks, our method consistently improves confidence-based fully non-AR decoding, outperforms explicit EOT suppression, and preserves the parallel decoding advantage of fully non-AR generation.