# DAPD：双锚定策略蒸馏缓解大模型后训练中的特权幻觉

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-03 08:00
- AIHOT 分数：52
- AIHOT 链接：https://aihot.virxact.com/items/cmseazfg10k6ero2ep4s4r7b8
- 原文链接：https://arxiv.org/abs/2608.01735

## AI 摘要

针对在线策略自蒸馏（OPSD）中因训练与推理信息不对称导致的特权幻觉问题，研究者提出双锚定策略蒸馏（DAPD）框架，通过双路径锚定与双源锚定防止特权依赖行为迁移至推理阶段。实验显示，DAPD 在 Qwen3-4B 上平均超越 OPSD 达 +2.00 分，且在 4B 与 32B 规模上分别提升 +2.69 与 +2.78 分。

## 正文

On-policy (self) distillation (OPSD) is increasingly adopted for language-model post-training. It strengthens the teacher with privileged information but can induce a privilege illusion: the student learns privilege-dependent behavior it cannot reproduce from its inference-time context, yet behaves as if the training-time privileged information remained available, ultimately degrading performance. In this paper, we identify information asymmetry between the privileged teacher and the student at inference as the root cause of this failure in OPSD. To resolve this asymmetry, we propose Dual-Anchored Policy Distillation (DAPD), a unified framework with two levels of anchoring. Dual-Path Anchoring (DPA) introduces a self-conditioned bridge and aligns reference and rollout behavior along two matched-information paths, preventing privilege-dependent behavior from being transferred to the inference-time student. Dual-Source Anchoring (DSA) applies these paths in both reference-to-rollout and rollout-to-reference directions, reducing reliance on privileged reference guidance while preserving correctness supervision. Extensive experiments show that DAPD significantly alleviates privilege illusion, outperforming OPSD on Qwen3-4B by +2.00 points on average across tasks. Notably, its gains persist across scales, reaching +2.69 at 4B and +2.78 at 32B.