# Latent-to-4D：从视频先验到 4D 世界的直接生成

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-11 08:00
- AIHOT 分数：49
- AIHOT 链接：https://aihot.virxact.com/items/cmspi2an8040kromi23qifbp1
- 原文链接：https://arxiv.org/abs/2608.10744

## AI 摘要

4D 生成通常依赖 RGB 视频重建或为特定视频生成器定制几何预测，两者分别受分布失配和生成器绑定限制。新方法 Latent-to-4D 直接对齐视频模型的去噪潜变量与预训练 4D 解码器的 token 网格，绕过 RGB 中间步骤，仅用约 1K 重建片段训练，单一 checkpoint 即可在同类 VAE 家族的不同视频扩散 Transformer 间迁移。

## 正文

4D generation synthesizes dynamic 3D scenes from conditions such as text or images. Existing methods either reconstruct generated RGB videos with a separate 4D model or adapt a particular video generator to predict geometry directly. The former suffers from distribution mismatch and error propagation, whereas the latter ties 4D prediction to a specific generator and may require retraining when the generator or conditioning regime changes. We ask whether the final denoised latents of video models that share a variational autoencoder (VAE) can instead provide a reusable interface to explicit 4D prediction. Building on this insight, we introduce direct latent-to-4D generation and instantiate it as Latent-to-4D, which bypasses RGB by aligning a video latent with the token grid of a pretrained 4D decoder and refining it through frame-wise and global spatiotemporal attention. Trained on roughly 1K existing reconstruction clips, a single checkpoint transfers unchanged across multiple video diffusion transformers within the same VAE family. On Text4D-200 and I4D-200, Latent-to-4D surpasses matched same-latent Wan+4RC cascades in projection-based DINO-F1 by 2.88--3.45 and 5.81 points, respectively, while also being preferred by human raters for geometry, temporal stability, and overall quality.