# RefCaptioner：多参考图像锚定的视频描述生成

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-07-30 08:00
- AIHOT 分数：50
- AIHOT 链接：https://aihot.virxact.com/items/cms8epxbt0auorot0otid0ucg
- 原文链接：https://arxiv.org/abs/2607.28509

## AI 摘要

RefCaptioner提出多参考图像锚定视频描述任务，通过两阶段训练框架联合优化参考选择、短语级绑定与干扰项拒绝。训练语料含20,000个视频和171,354张参考图像，并引入MRVBench基准评测。实验显示其在开源模型中综合表现最佳，人工评估中描述更受偏好。

## 正文

Existing video captioning models generate natural descriptions of video content but cannot explicitly ground local visual elements to multiple reference images. We introduce multi-reference image-grounded video captioning, a new task requiring factual video descriptions with phrase-level reference grounding, and propose RefCaptioner, a two-stage post-training framework for this task. RefCaptioner combines mixed-data SFT with Hierarchical Coverage-Discounted GRPO to jointly improve reference selection, phrase-level binding, distractor rejection, and cross-reference consistency while preserving general video-captioning ability. To support training, we construct a corpus containing $20,000$ videos and 171,354 reference images. We further introduce MRVBench, a benchmark for evaluating caption factuality and multi-reference grounding on both real-world and AI-generated videos. Experiments show that RefCaptioner achieves the best overall performance among the open-source models while remaining competitive on standard video captioning benchmarks. Human evaluation further confirms that its captions are preferred by annotators and enable more source-faithful video reconstruction with both open-source and proprietary video generators.