# 从2D网格到1D token：改革多模态图像融合的共享表示

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-10 08:00
- AIHOT 分数：59
- AIHOT 链接：https://aihot.virxact.com/items/cmqap4xzc0njcslldf4ajjhgu
- 原文链接：https://arxiv.org/abs/2606.12303

## AI 摘要

多模态图像融合现有方法基于2D特征网格，局部建模强但全局外观控制有限。本文引入紧凑1D token接口，基于冻结预训练图像tokenizer作为全局载体，同时保留2D空间路径恢复局部结构。提出选择性token编辑（STE），稀疏更新或替换关键token，在不改变融合主干、不引入额外损失下引导全局一致性。在四个基准上取得最佳整体性能，全局一致性和局部保真度均提升。

## 正文

Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image-level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non-local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi-metric improvements in both global coherence and local fidelity. Project page: https://zju-xyc.github.io/1D-Fusion-Project-Page/