Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposition is essential for stability, extreme decomposition creates a “no-recovery bottleneck”. We show that this bottleneck becomes critical due to highly non-uniform error distribution, where consistent errors on a few “hard” steps become irreversible. To address this, we propose Lookahead-Enhanced Atomic Decomposition (LEAD). By incorporating short-horizon future validation and aggregating…
Apple 提出 LEAD 方法,破解长程推理中的"不可恢复瓶颈"
AI 导读
Apple 研究发现,大语言模型在长程执行中即使有高层策略也不稳定,极端分解会导致“不可恢复瓶颈”——少数“困难”步骤上的持续错误变得不可逆转。为此提出 Lookahead-Enhanced Atomic Decomposition(LEAD),通过引入短程未来验证与聚合来打破这一瓶颈。该方法在受控算法谜题上验证了有效性。
Apple Machine Learning Research(RSS)
51
AI 编辑部评分,满分 100Apple 提出 LEAD 方法,破解长程推理中的"不可恢复瓶颈"
Apple 研究发现,大语言模型在长程执行中即使有高层策略也不稳定,极端分解会导致“不可恢复瓶颈”——少数“困难”步骤上的持续错误变得不可逆转。为此提出 Lookahead-Enhanced Atomic Decomposition(LEAD),通过引入短程未来验证与聚合来打破这一瓶颈。该方法在受控算法谜题上验证了有效性。
原文 · 保持原样,未翻译原文 · 未翻译
来源:Apple Machine Learning Research(RSS)· machinelearning.apple.com