VGenST-Bench:一个基于主动视频合成的时空推理基准
阅读原文· arxiv.orgVGenST-Bench 是一个用于评估多模态大语言模型时空推理能力的新基准,采用生成模型主动合成高度可控的多样化评估场景,克服了现有基准依赖静态图像或被动视频数据的局限。该基准通过包含人类质检阶段的多智能体流水线构建,建立了涵盖空间尺度、视角和场景动态性的 3x2x2 视频分类体系。其设计的层级任务套件,解耦了低级视觉感知与高级时空推理,实现了对模型能力的细粒度诊断。
Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-temporal reasoning benchmark datasets primarily rely on static image sets or passively curated video data, which limits the evaluation of fine-grained reasoning capabilities. In this paper, we introduce VGenST-Bench, a video benchmark that employs generative models to actively synthesize highly controlled and diverse evaluation scenarios. To construct VGenST-Bench, we propose a multi-agent pipeline incorporating a human quality control stage, ensuring the quality of all generated videos and QA pairs. We establish a comprehensive 3x2x2 video taxonomy, encompassing Spatial Scale, Perspective, and Scene Dynamics to span diverse scenarios. Furthermore, we design a hierarchical task suite that decouples low-level visual perception from high-level spatio-temporal reasoning. By shifting the paradigm from passive curation to active synthesis, VGenST-Bench enables fine-grained diagnosis of spatio-temporal understanding in MLLMs.