Puffin-World 提出原生 3D 世界状态的统一多模态世界模型并开源

HuggingFace Daily Papers(社区热门论文)·2026-09-03 08:00·1天前
AI 导读

Puffin-World 是一个统一多模态架构,不依赖外部离线模块即可实现物理理解、空间模拟和 3D 世界生成与重建。该框架联合建模物理(重力场与纬度)、几何(深度)和外观(图像)三种原生世界状态,并引入统一 Omni-Camera 表示与跨未来帧传播物理动态的策略,在单一生成过程中同时合成未来视图并重建底层几何。

HuggingFace Daily Papers(社区热门论文)
41AI 编辑部评分,满分 100

Puffin-World 提出原生 3D 世界状态的统一多模态世界模型并开源

2026-09-03 08:00· 1天前
AI 导读

Puffin-World 是一个统一多模态架构,不依赖外部离线模块即可实现物理理解、空间模拟和 3D 世界生成与重建。该框架联合建模物理(重力场与纬度)、几何(深度)和外观(图像)三种原生世界状态,并引入统一 Omni-Camera 表示与跨未来帧传播物理动态的策略,在单一生成过程中同时合成未来视图并重建底层几何。

We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules. To reliably construct and interact with 3D worlds, our framework jointly models three native world states: physics (gravity field and latitude), geometry (depth), and appearance (image), together with a unified Omni-Camera representation that supports diverse tasks and flexible motions. Beyond modeling these states, we introduce a strategy for propagating physical dynamics across future frames. By grounding absolute camera properties in the real world, Puffin-World enables physically consistent and visually stable world generation. We further couple appearance and geometry within a single generative process, jointly synthesizing each future view and reconstructing its underlying geometry. This unified paradigm enables interleaved closed-loop applications requiring synergy across multiple tasks, including mimic and self-calibrated world exploration. To scale Puffin-World to complex scenarios, we construct Puffin-16M, comprising 15 million vision-language-camera triplets and 1 million trajectories featuring various and challenging motions. To foster further research in this area, we released the code, models, and datasets.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org