理解环境感知信息检索的行为:强化学习如何为不同检索器定制查询策略
阅读原文· arxiv.org基于检索增强生成(RAG)的方法在处理复杂查询时表现出色,但不同检索器需要截然不同的查询构建策略。该工作首次系统分析大语言模型(LLM)如何通过强化学习(RL)学习为不同检索器定制查询策略。实验表明,RL能有效指导LLM根据检索器特性调整查询风格,且不同检索器对描述型或疑问型查询的偏好显著不同。引入分支式rollout技术提升了多步检索轨迹下的训练稳定性。研究为构建真正检索器感知的RAG系统提供了实证证据和可操作洞见。代码与资源已公开。
Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.