# Imaginative Perception Tokens 增强多模态语言模型的空间推理

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-03 08:00
- AIHOT 分数：55
- AIHOT 链接：https://aihot.virxact.com/items/cmq5khxtw00hbsl38vfpfi3ac
- 原文链接：https://arxiv.org/abs/2606.03988

## AI 摘要

多模态语言模型在无法直接观测空间信息时推理能力不足。Imaginative Perception Tokens (IPT) 是一种中间感知表征，使模型能在保持与输入观测一致的前提下，外推出替代空间配置下的感知结果。研究基于统一 VLM 模型 BAGEL，构建了 Perspective Taking、Path Tracing 和 Multiview Counting 三个任务共约 2 万样本的基准。IPT 监督训练持续提升空间推理性能，在 MVC 上准确率提升 3.4%，在 PT 上与强闭源模型相当，且常优于文本思维链训练。IPT 为不可观测空间结构提供了原则性监督信号，同时生成可解释的中间表征。

## 正文

Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable. Many such problems require imaginative perception: inferring what would be seen from an unseen viewpoint, tracing paths through occluded spaces, or integrating partial observations into a coherent spatial representation. We introduce Imaginative Perception Tokens (IPT), intermediate perceptual representations that externalize what a VLM would perceive under alternative spatial configurations while remaining consistent with the observed input. To study this capability, we formulate three tasks, Perspective Taking (PET), Path Tracing (PT), and Multiview Counting (MVC), and construct datasets of approximately 20K examples with ground truth imaginations, answers, and evaluation benchmarks. Using the unified VLM BAGEL as the backbone, IPT supervision consistently improves spatial reasoning and often outperforms textual chain of thought training, even without generating images at inference time. On MVC, IPT improves accuracy by 3.4% and achieves competitive performance with strong closed-source models on PT. We further find that combining IPT and label-only supervision yields additional gains, whereas textual chain of thought can substantially degrade performance, suggesting a modality mismatch when spatial computation is forced through language. Overall, IPT provides a principled supervision signal for reasoning about unobserved spatial structure, improving generalization while producing interpretable intermediate representations.