# OmniAgent：原生全模态智能体实现长视频主动感知推理

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-17 08:00
- AIHOT 分数：65
- AIHOT 链接：https://aihot.virxact.com/items/cmqj0n3n60625sl5wkf02dv7o
- 原文链接：https://arxiv.org/abs/2606.19341

## AI 摘要

OmniAgent 提出首个原生全模态智能体框架，将长视频理解建模为基于 POMDP 的迭代观察-思考-行动循环。它通过按需动作选择性提取音视频线索并转化为持久文本记忆，使推理复杂度与视频时长解耦。训练采用 Agentic SFT（最佳轨迹合成与双阶段质量控制）和基于 TAURA 的 Agentic RL（利用 turn 级熵分配探索奖励）。模型在测试时呈现正向缩放：推理轮次越多性能越强。在 VideoMME、LVBench 等 10 项基准上，OmniAgent 达到开源模型最佳水平。7B 参数版本在 LVBench 上以 50.5% 超越 10 倍大的 Qwen2.5-VL-72B（47.3%）。

## 正文

Passive models for long video understanding typically rely on a "watch-it-all" paradigm, processing frames uniformly regardless of query difficulty, causing computational cost to grow with video duration. Although interactive frameworks have emerged, they often rely on global pre-scanning, and their context cost still scales with video length. We propose OmniAgent, the first native omni-modal agent that formulates video understanding as a POMDP-based iterative Observation-Thought-Action cycle. OmniAgent executes on-demand actions to selectively distill audio-visual cues into a persistent textual memory, effectively decoupling reasoning complexity from raw video duration. To operationalize this, we introduce (1) Agentic Supervised Fine-Tuning to bootstrap native active perception via best-of-N trajectory synthesis with dual-stage quality control, and (2) Agentic Reinforcement Learning with TAURA (Turn-aware Adaptive Uncertainty Rescaled Advantage), which leverages turn-level entropy to steer credit assignment toward pivotal discovery turns. Crucially, OmniAgent exhibits positive test-time scaling, where performance improves as the number of reasoning turns increases, validating the efficacy of active perception. Empirical results across ten benchmarks (e.g., VideoMME, LVBench) demonstrate that OmniAgent achieves state-of-the-art performance among open-source models. Notably, on LVBench, our 7B agent outperforms the 10times larger Qwen2.5-VL-72B (50.5% vs. 47.3%).