# Lip Forcing：用于实时唇同步的少步自回归扩散方法

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-10 01:56
- AIHOT 分数：64
- AIHOT 链接：https://aihot.virxact.com/items/cmq7nnzgf001hslnklwj1b3iq
- 原文链接：https://arxiv.org/abs/2606.11180

## AI 摘要

Lip Forcing提出了自回归扩散方法用于视频到视频唇同步，从14B参数的音频条件双向视频扩散教师模型蒸馏出因果学生模型。推理时每个块仅需两步去噪，无需CFG，实现实时流式处理。技术分析揭示CFG的保真度-同步权衡，进而衍生出Sync-Window DMD、两步推理调度和基于SyncNet的奖励三项组件。1.3B学生模型在31 FPS下实时输出，比同规模双向模型快17.6倍；14B学生模型是目前最大的V2V唇同步扩散模型，比教师快39.8倍，保真度接近。首帧时延均小于1毫秒。

## 正文

Diffusion-based lip synchronization models achieve strong visual quality and audio-visual alignment, but full-sequence bidirectional attention and many denoising steps make them impractical for real-time inference. We present Lip Forcing, to our knowledge the first autoregressive diffusion method for video-to-video (V2V) lip synchronization, which distills a 14B audio-conditioned bidirectional video diffusion teacher into causal students. At inference, the students generate each chunk in only two denoising steps without inference-time CFG, enabling real-time lip synchronization. A lip-sync-specific teacher-trajectory analysis reveals a CFG fidelity-sync tradeoff: no-CFG predictions favor reference fidelity, whereas CFG-guided predictions favor synchronization within a mid-trajectory band. Lip Forcing translates this finding into three analysis-derived components: Sync-Window DMD, a two-step inference schedule, and a SyncNet-based reward. We validate Lip Forcing at two student scales, both distilled from the 14B teacher. The 1.3B student crosses into real-time streaming at 31 FPS, 17.6times faster than its same-scale bidirectional model. The 14B student, the largest diffusion model reported for V2V lip synchronization, runs 39.8times faster than its teacher at comparable reference fidelity. Time-to-first-frame is sub-millisecond at both scales, far below every diffusion baseline.