# ViQ：任意分辨率的文本对齐视觉量化表示

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-25 08:00
- AIHOT 分数：54
- AIHOT 链接：https://aihot.virxact.com/items/cmqwqyy1k006wslikgg485xq5
- 原文链接：https://arxiv.org/abs/2606.27313

## AI 摘要

ViQ 是一种视觉量化表示框架，通过两阶段学习（文本对齐预训练与特征离散化）在离散表示中平衡语义与细节，并支持原生分辨率输入。预训练借助语言模型增强语义监督，离散化阶段采用近端表示学习逐步压缩特征空间，结合位置感知多头量化实现任意分辨率处理。多模态任务上，ViQ 达到与基于连续高维特征的 SOTA 编码器相当的竞争力，同时保持低层重建高精度。采用 ViQ 的量化表示进行多模态训练可实现 20%–70% 的加速，适用于不同大语言模型和训练方案。

## 正文

A unified representation for text and vision is a natural pursuit, as it enables simpler multimodal modeling and more efficient training. However, representing images as discrete signals in the same way as text inevitably introduces severe information loss. Existing work struggles to balance low-level details and high-level semantics in discrete representations: reconstruction-oriented representations often lack semantic information, whereas semantically stronger features typically suffer from severe loss of detail. We present ViQ, a Visual Quantized Representations framework, which is designed to balance semantics and details in discrete representations while supporting inputs at native resolutions, thereby enabling it to serve as a unified and general discrete representation for arbitrary visual inputs. Our approach structures quantization learning into two stages: text-aligned pre-training and feature discretization. With text-aligned pre-training, we enhance the visual encoder semantic-rich supervision from the pretrained language model and enable it to process native-resolution visual inputs. During discretization, we propose a proximal representation learning strategy to progressively compact the feature space, along with a position-aware head-wise quantization mechanism that enables flexible processing of arbitrary resolutions. Extensive experiments on multimodal tasks demonstrate that ViQ achieves competitive performance compared to state-of-the-art multimodal vision encoders with continuous and high-dimensional visual features, while maintaining high precision in low-level reconstruction. We also show that multimodal training with visual quantized representations largely improves efficiency, yielding up to 20\%-70\% acceleration with different base LLMs and training recipes.