Apple Machine Learning Research（RSS）

Apple 提出 SRLM：自反思程序搜索提升长上下文处理能力

2026-07-09 08:00·1天前

AI 摘要

Apple 机器学习研究团队提出 SRLM 框架，利用自一致性、推理链长度和口头置信度三种内在信号，让模型在推理时评估候选长上下文交互程序。实验表明，在相同时间预算下，SRLM 较传统递归语言模型（RLM）最高提升 22%。分析发现，递归本身并非 RLM 性能关键，简单的自反思程序搜索无需显式递归即可匹配或超越 RLM；在模型上下文窗口内，RLM 反而降低性能，而 SRLM 在短上下文和长上下文中均实现稳定增益。

原文 · 未翻译

research area Speech and Natural Language Processing

content type paperpublished July 2026

Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context

AuthorsKeivan Alizadeh*, Parshin Shojaee*, Minsik Cho, Mehrdad Farajtabar

View publication

Long-context handling remains a core challenge for language models: even with extended context windows, models often fail to reliably extract, reason over, and use the information across long contexts. Recent works like Recursive Language Models (RLMs) have approached this challenge by agentic way of decomposing long contexts into recursive sub-queries through programmatic interaction at inference. While promising, the success of RLMs critically depends on how these trajectories of context-interaction programs are selected, which has remained unexplored. In this paper, we study this problem and introduce Self-Reflective Program Search for Long Context (SRLM), a framework that augments programming-based context interaction with uncertainty-aware self-reflection. SRLM leverages three intrinsic signals: self-consistency, reasoning trace length, and verbalized confidence. These serve as complementary indicators of a model’s internal uncertainty, and the model uses them to evaluate and compare candidate context-interaction programs. Extensive experiments across diverse benchmark datasets, context lengths, and backbone models, show that SRLM consistently outperforms state-of-the-art baselines, yielding up to 22% improvement over RLMs under the same time budget. Our findings show that recursion itself is not the primary driver of performance in RLMs, and a simple self-reflective program search can match or surpass RLM without requiring self-query or explicit recursion mechanisms. We find that for context lengths within the model’s context window, RLMs with recursion often degrade performance relative to the base model, whereas SRLM yields consistent and robust gains across both short and long contexts. We also find that RLM is less effective in tasks with semantically intensive nature, where heuristic program search is insufficient and broader contextual understanding is required, while self-reflection in SRLM provides a semantic signal that better steers reasoning in these challenging long-context scenarios.