# UniDoc-RL：基于层次化动作与密集奖励的由粗到细视觉RAG

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-16 08:00
- AIHOT 链接：https://aihot.virxact.com/items/cmo2bd9a5022oslba8mee8pnq
- 原文链接：https://arxiv.org/abs/2604.14967

## AI 摘要

针对现有视觉RAG系统忽略细粒度视觉语义的问题，本文提出UniDoc-RL统一强化学习框架。该方法将视觉信息获取建模为层次化顺序决策过程，通过从粗粒度文档检索到细粒度图像选择再到主动区域裁剪的渐进式策略，使大型视觉语言模型智能体联合执行检索、重排序与推理。引入密集多奖励方案为每个动作提供任务感知监督，并基于GRPO算法实现端到端训练而无需价值网络。在三个基准测试中，该方法相比现有基于RL的方法性能提升最高达17.7%。

## 正文

Retrieval-Augmented Generation (RAG) extends Large Vision-Language Models (LVLMs) with external visual knowledge. However, existing visual RAG systems typically rely on generic retrieval signals that overlook the fine-grained visual semantics essential for complex reasoning. To address this limitation, we propose UniDoc-RL, a unified reinforcement learning framework in which an LVLM agent jointly performs retrieval, reranking, active visual perception, and reasoning. UniDoc-RL formulates visual information acquisition as a sequential decision-making problem with a hierarchical action space. Specifically, it progressively refines visual evidence from coarse-grained document retrieval to fine-grained image selection and active region cropping, allowing the model to suppress irrelevant content and attend to information-dense regions. For effective end-to-end training, we introduce a dense multi-reward scheme that provides task-aware supervision for each action. Based on Group Relative Policy Optimization (GRPO), UniDoc-RL aligns agent behavior with multiple objectives without relying on a separate value network. To support this training paradigm, we curate a comprehensive dataset of high-quality reasoning trajectories with fine-grained action annotations. Experiments on three benchmarks demonstrate that UniDoc-RL consistently surpasses state-of-the-art baselines, yielding up to 17.7% gains over prior RL-based methods.