# 用于生成的原生音视频对齐

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-05-28 08:00
- AIHOT 分数：57
- AIHOT 链接：https://aihot.virxact.com/items/cmpqb0qot03dvslno7qb3vlzi
- 原文链接：https://arxiv.org/abs/2605.30073

## AI 摘要

针对现有开源方法在联合音视频生成中存在的音视频精细协同不足或语义条件与底层同步耦合的问题，本文提出了NAVA（原生音视频对齐）框架。该框架在专用交互空间建立音视频对应关系后，利用外部上下文条件化联合去噪过程。具体实现上，NAVA采用Align-then-Fuse MMDiT架构，并引入Timbre-in-Context Conditioning技术。在Verse-Bench和Seed-TTS上的实验表明，NAVA仅用6.3B参数即实现了高质量的视频生成、精准的音视频同步、有竞争力的音频质量以及更强的参考音色可控性。

## 正文

Joint audio-video generation aims to synthesize temporally synchronized and semantically coherent visual-acoustic content. However, existing open-source methods mainly rely on either dual-tower designs with posterior alignment or fully unified tri-modal designs that mix textual context, audio and video in one shared space. The former weakens fine-grained audio-video co-evolution, while the latter couples semantic conditioning with low-level synchronization. To address these limitations, we propose NAVA, a Native Audio-Visual Alignment framework for joint audio-video generation. NAVA is built upon context-conditioned native audio-visual alignment: it first establishes audio-video correspondence in a dedicated interaction space, and then uses external context to condition the joint denoising process. Specifically, NAVA is instantiated with an Align-then-Fuse MMDiT architecture, which transitions from modality-aware audio-video alignment to modality-shared joint denoising. Furthermore, we introduce Timbre-in-Context Conditioning to associate reference timbre cues with corresponding speech spans to achieve controllable speech timbre. Experiments on Verse-Bench and Seed-TTS, together with a user study, demonstrate that NAVA achieves superior video quality, precise audio-visual synchronization, competitive audio quality, and stronger reference-timbre controllability using only 6.3B parameters.