# 基于通用关键帧提取连接视频问答与视频引导智能体任务

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-28 08:00
- AIHOT 分数：55
- AIHOT 链接：https://aihot.virxact.com/items/cmr00z6hf02uwslki0o92n5h6
- 原文链接：https://arxiv.org/abs/2606.29445

## AI 摘要

论文提出VG-GUIBench基准，用于评估多模态大语言模型（MLLM）的GUI智能体能否跟随视频教程完成交互任务。现有VideoQA基准侧重浅层视觉线索，而VG-GUIBench考察模型从视频中学习深层知识并泛化到长时智能体任务。同时提出TASKER关键帧提取算法，联合考虑任务相关性与场景动态筛选信息帧。实验显示，TASKER在EgoSchema全集上超出最优基线2.0%，在NExT-QA数据集上超出1.8%，展示了通用关键帧提取方法在视频理解任务中的潜力。代码与数据已公开。

## 正文

Video understanding is a fundamental capability for multimodal intelligence, and recent Multimodal Large Language Models (MLLMs) have achieved remarkable performance on Video Question Answering (VideoQA) benchmarks. However, existing benchmarks primarily evaluate whether models can perceive shallow visual cues, while rarely examining whether MLLMs can learn deeper knowledge or procedural skills from video tutorials and generalize them to downstream long-horizon agentic tasks. To address this gap, we introduce VG-GUIBench (Video-Guided GUI Benchmark), a new benchmark designed to evaluate whether MLLM-based GUI agents can follow video tutorials to complete corresponding GUI interactive tasks. Furthermore, we observe that the performance of models on both VideoQA and video-guided agentic tasks critically depends on effective keyframe extraction. Based on this observation, we propose TASKER (Task-driven And Scene-aware Keyframe searchER), a keyframe extraction algorithm that jointly considers task relevance and scene dynamics to identify informative frames. Experimental results demonstrate that TASKER achieves significant performance improvements on both VideoQA and video-guided agentic task benchmarks, outperforming the best baseline by 2.0% on the EgoSchema fullset and 1.8% on the NExT-QA dataset, respectively. These results further highlight the potential of generalized keyframe extraction methods for video understanding tasks. Our code and data are available at https://github.com/VG-GUI-TASKER/VG-GUI-TASKER.