# KVAE：面向多模态生成模型的 tokenizer 系列

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-06 08:00
- AIHOT 分数：51
- AIHOT 链接：https://aihot.virxact.com/items/cmsjb2bgt04w4roo54k6nb475
- 原文链接：https://arxiv.org/abs/2608.05798

## AI 摘要

KVAE 系列 tokenizer 覆盖音频、图像与视频，专为后续文本条件生成设计。其中 KVAE-Audio 为连续全频带 48 kHz tokenizer，具备 50 Hz 潜空间与 64 通道；KVAE-3D 提供 4x16x16 与 4x8x8 两种因果视频压缩方案；KVAE-2D 图像模型实现 8 倍压缩与 32 通道。

## 正文

Latent diffusion modeling (LDM), a prominent paradigm, utilizes tokenizers to map input signal to compressed representation. This dependency positions tokenizer as an integral part of generation process itself, since it affects learning speed, quality of synthesized samples and lay foundation for later applications. This report presents series of KVAE tokenizers for audio, image and video, all designed for subsequent text-conditioned generation: KVAE-Audio, a continuous full-band 48 kHz tokenizer with a 50 Hz latent of 64 channels; KVAE-3D -- two causal video tokenizers for 4x16x16 and 4x8x8 compression; KVAE-2D, an image model, compressing input by factor of 8 with 32 channels. We demonstrate that reconstruction (PSNR, LPIPS, PESQ, etc.) and generation results on objective (Frechet Distance, CLIP score, CLAP score, etc.) and subjective (side-by-side evaluation) metrics matches or surpasses frontier opensource tokenizers, such as VAEs from Wan-2.2, HunyuanVideo-1.5, FLUX.2, MovieGen, StableAudio and MMAudio. Considering difficulty of development, we share with community training details, model selection method and ablation on design choices. The code is publicly available at https://github.com/kandinskylab/kvae and https://github.com/kandinskylab/kvae-audio.