Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos. Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent. At inference, IVT generates the answer directly without synthesizing or re-encoding future frames. We conduct controlled studies across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives. IVT improves over direct-answer fine-tuning on all six evaluation settings while retaining the same inference pathway. Compared with explicit Visual CoT, IVT achieves comparable or better performance and reduces average end-to-end latency by more than 5x. Together, our findings suggest that explicit pixel-space generation at inference time, as used in visual chain-of-thought, may not be necessary for effective proactive video reasoning. Predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.
超越视觉思维链:面向主动视频推理的内化视觉思维
AI 导读
新提出的IVT后训练框架在无标注视频上联合优化文本预测与下一嵌入预测,使模型推理时直接生成答案,无需合成或重编码未来帧。相比显式视觉CoT,IVT性能相当或更优,端到端延迟平均降低5倍以上,并在全部六项评测设置上优于直接答案微调。
HuggingFace Daily Papers(社区热门论文)
55
AI 编辑部评分,满分 100超越视觉思维链:面向主动视频推理的内化视觉思维
新提出的IVT后训练框架在无标注视频上联合优化文本预测与下一嵌入预测,使模型推理时直接生成答案,无需合成或重编码未来帧。相比显式视觉CoT,IVT性能相当或更优,端到端延迟平均降低5倍以上,并在全部六项评测设置上优于直接答案微调。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org