# 长程规划智能体：预训练与OPD蒸馏研究

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-08-25 07:33
- AIHOT 分数：34
- AIHOT 链接：https://aihot.virxact.com/items/cmt7vztt32jk3ro73hp5czw2o
- 原文链接：https://x.com/rohanpaul_ai/status/2092032639318569357

## AI 摘要

论文发现，后训练无法修复薄弱的长程规划基础：噪声轨迹会累积错误，稀疏奖励导致信用分配不当，冲突教师引发遗忘。研究建议优先训练清晰世界模型和长轨迹，奖励信号过稀疏时采用OPD（on-policy distillation），并避免合并规划策略不兼容的教师。OPD在长程噪声场景下优于结果奖励GRPO，因教师反馈贯穿整个轨迹而非仅在末端。

## 正文

No amount of post-training cleanly fixes weak long-horizon foundations: noisy trajectories compound errors, sparse rewards misassign credit, and conflicting teachers trigger forgetting.

This paper finds, ff you want agents to stay reliable over long tasks, give them clean world-model and long-trajectory training first, use OPD (on-policy distillation) when reward signals get too sparse, and avoid merging teachers with incompatible planning strategies.

Suboptimal trajectories were especially damaging because small mistakes accumulated until middle and long tasks nearly collapsed.

For post-training, OPD handled longer, noisier settings better than outcome-reward GRPO because teacher feedback arrived throughout the trajectory instead of only at the end.

– arxiv. org/abs/2607.24720v1

Title: "The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation"