为何远处看向上方:探查视觉语言模型中的空间表征
阅读原文· arxiv.org视觉语言模型(VLMs)在空间推理基准上表现优异,但其理解是否基于真正的3D结构尚不明确。研究通过构建对比嵌入对进行表征分析,发现多个模型族存在一致的“垂直距离纠缠”现象,即模型将图像垂直位置与空间距离混淆,这模仿了自然照片的透视偏差。该偏差导致模型在透视一致与反直觉案例间准确率差距显著,且随数据规模扩大而加剧,即使基准分数提升。分析还表明,基准分数相似的模型可能具有不同的内部表征,这能预测其在不同任务中的准确率与鲁棒性。为隔离数据集偏差,团队推出了合成基准SpatialTunnel,实验证实该纠缠是模型固有属性,空间轴分离度更高的模型表现更鲁棒。
Vision-language models (VLMs) achieve strong performance on spatial reasoning benchmarks, yet it remains unclear whether this reflects structured 3D understanding or reliance on statistical shortcuts in natural images. We introduce a representation-level analysis framework that constructs minimal contrastive pairs to measure how spatial axes are organized and disentangled within VLM embeddings. Our analysis across multiple model families reveals a consistent vertical-distance entanglement: models conflate vertical image position with distance, mirroring the perspective bias of natural photographs. This bias produces a significant accuracy gap between perspective-consistent and counter-heuristic examples, and intensifies under data scaling even as overall benchmark accuracy improves. We further show that models with similar benchmark scores can exhibit different internal representations, and that these differences predict accuracy and robustness across diverse spatial reasoning benchmarks. To isolate this bias from evaluation-set skew, we introduce SpatialTunnel, a synthetic benchmark designed to expose spatial shortcut biases by removing common correlations present in natural images. Experiments confirm that the entanglement is model-intrinsic, and that models with well-separated spatial axes exhibit greater robustness, suggesting that well-structured spatial representations lead to more reliable spatial reasoning across diverse benchmarks. Code and benchmark are available on the project page: https://cheolhong0916.github.io/whyfarlooksup.github.io/.