# NVIDIA OmniDreams：用于闭环自动驾驶模拟的实时生成式世界模型

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-02 08:00
- AIHOT 分数：65
- AIHOT 链接：https://aihot.virxact.com/items/cmpxgncg903s0slcktz34ny4w
- 原文链接：https://arxiv.org/abs/2606.03159

## AI 摘要

OmniDreams是一个基于Cosmos扩散模型进行中后期训练的基础生成式世界模型，使用21k小时驾驶场景数据训练。它能根据过去帧、当前仿真器状态和即时驾驶动作，自回归地实时生成动作条件化的逼真传感器视频，可合成极端天气和不可预测的动态智能体行为等复杂现象。该模型部署于包含Alpamayo 1策略模型和AlpaSim协调器的闭环系统中，作为高响应性环境。初步结果显示，基于OmniDreams后训练的世界-动作模型（WAM）在Physical AI自动驾驶NuRec数据集上，仅用Alpamayo 1.5研究策略模型1/5的总参数就取得了优异性能。

## 正文

As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations. While recent reconstruction-based neural simulators offer photorealism, they are fundamentally constrained by their initial captured data and struggle to generalize to highly dynamic or novel scenes. To overcome these limitations, we introduce OmniDreams, a foundation generative world model mid- and post-trained from the Cosmos diffusion model to autoregressively generate action-conditioned videos in real time. By leveraging the rich visual priors of Cosmos and mid- and post-training on 21k hours of driving scenarios, OmniDreams synthesizes complex, unobserved phenomena that are hard for traditional simulators to capture, such as extreme weather and unpredictable dynamic agent behaviors. Crucially, it autoregressively conditions its photorealistic sensor generation on past frames, the current simulator state, and immediate driving actions. Deployed in a closed-loop system with the Alpamayo 1 policy model and AlpaSim orchestrator, OmniDreams acts as a highly responsive, reactive environment, providing a scalable and comprehensive solution for training and evaluating next-generation autonomous driving policies. We additionally show preliminary results indicating that a world-action model (WAM) post-trained from OmniDreams achieves strong performance on the Physical AI Autonomous Vehicles NuRec dataset, surpassing the VLA-based Alpamayo 1.5 research policy model while using only 1/5 the total parameters. These results highlight the potential for a real-time world model like OmniDreams to also serve as a backbone for policy architectures.