SpatialWorld:多模态智能体在真实世界任务中的交互式空间推理基准
阅读原文· arxiv.orgSpatialWorld 是为评估多模态智能体在复杂真实任务中的交互式空间理解而设计的统一基准。它整合 8 个异构模拟后端,包含 760 个人工标注任务,覆盖家庭日常、旅行、社交协作等场景。智能体需在仅视觉部分可观测条件下主动收集第一人称证据,并通过统一文本动作接口输出决策。对 15 个先进多模态智能体的评测显示,最强闭源模型 GPT-5 平均任务成功率仅 17.4%,最强开源模型 Qwen-3.5 达 14.1%。分析表明任务成功与执行效率存在脱节,不同领域性能差异显著,主动探索与长程规划仍是瓶颈。
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predominantly rely on passive evaluation (e.g., static VQA) or simulator-specific pipelines, failing to assess general interactive spatial understanding. We introduce SpatialWorld, a unified benchmark designed specifically for evaluating the interactive spatial understanding of multimodal agents in complex real-world tasks. Integrating eight heterogeneous simulation backends under a shared, simulator-agnostic protocol, SpatialWorld features 760 human-annotated tasks across diverse domains (e.g., household routines, travel, social collaboration). Agents must solve tasks under vision-only partial observability, actively gathering egocentric visual evidence and expressing decisions via a unified, text-based action interface native to MLLMs. For reliable evaluation, each task includes a human-validated initial state, a reference trajectory, and a terminal-state verifier. Evaluating 15 advanced agents reveals that robust spatial task solving remains challenging: the strongest model, GPT-5, achieves an average task success rate (TSR) of only 17.4%, while the leading open-source model, Qwen-3.5, reaches 14.1%. Further analysis exposes a clear mismatch between task success and execution efficiency, alongside substantial domain-specific performance variations. These bottlenecks in active exploration and long-horizon planning position SpatialWorld as a rigorous testbed for future spatial agents.