# 伯尼尼：基于潜在语义规划的视频扩散模型

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-05-21 08:00
- AIHOT 分数：60
- AIHOT 链接：https://aihot.virxact.com/items/cmpgadnmr0dqqsljwxjze5ncc
- 原文链接：https://arxiv.org/abs/2605.22344

## AI 摘要

本文提出了Bernini，一个用于视频生成与编辑的统一框架。该框架创新性地将多模态大语言模型与扩散模型分工协作：MLLM负责在ViT嵌入空间预测目标语义表示，扩散模型则依据此语义规划及文本特征合成像素。为处理多视觉输入，模型引入了分段感知三维旋转位置编码，并结合思维链推理，显著提升了从理解到生成的转化能力。该架构支持模块化训练与轻量协同优化，在多项视频生成与编辑基准测试中均取得最优表现。

## 正文

Multimodal large language models (MLLMs) and diffusion models have each reached remarkable maturity: MLLMs excel at reasoning over heterogeneous multimodal inputs with strong semantic grounding, while diffusion models synthesize images and videos with photorealistic fidelity. We argue that these two families can be unified through a simple division of labor: MLLMs perform semantic planning, while diffusion models render pixels from high-level semantic guidance and low-level visual features. Building on this idea, we propose Bernini, a unified framework for video generation and editing. An MLLM-based planner predicts the target semantic representation directly in the ViT embedding space, and a DiT-based renderer synthesizes pixels conditioned on this plan, augmented by text features and, for editing, source VAE features for detail preservation. Because semantics serve as the interface, the planner and renderer can be trained separately and only lightly co-trained, preserving the pretrained strengths of both components while keeping training efficient. To better handle multiple visual inputs, we introduce Segment-Aware 3D Rotary Positional Embedding (SA-3D RoPE), and further incorporate chain-of-thought reasoning in the planner to better transfer understanding into generation. Bernini achieves state-of-the-art performance across a wide range of video generation and editing benchmarks, with the MLLM's pretrained understanding translating into strong generalization on challenging editing tasks.