# Dream.exe：视频生成模型能否生成可执行的机器人操作？

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-04 08:00
- AIHOT 分数：63
- AIHOT 链接：https://aihot.virxact.com/items/cmq0suz4j09a5sltru1hzsbao
- 原文链接：https://arxiv.org/abs/2606.04811

## AI 摘要

Dream.exe 是一个评估视频生成模型物理世界理解能力的框架，通过视频到执行的流水线，将生成的机器人操作视频转化为真实可执行轨迹，并在物理模拟器中验证成功率。研究评估了8个模型，包括GPT-4o、Sora、CogVideoX-5B、Meta-LLaMA 3D、GR-1等，覆盖101个手动设计的操作任务，分为三个物理复杂度等级，评测视觉质量、轨迹保真度和执行成功率。结果显示，多个模型达到了可测量的执行成功率，但视觉质量不能预测可执行性。Dream.exe即将开源。

## 正文

Video generation models have made impressive strides in synthesizing visually compelling content, yet their outputs remain confined to the virtual domain. A natural question follows: how well do these models reflect the physical world when their generated videos leave the screen and enter reality? We propose robotic manipulation as a concrete, measurable window onto this question: if a model has truly internalized physical laws, the motion it depicts should translate into executable robot behavior. We introduce Dream.exe, an evaluation framework that operationalizes this criterion through a video-to-execution pipeline. Given a scene image and a task description, Dream.exe synthesizes a manipulation video, converts the generated motion into robot trajectories, and executes them in a physics simulator, yielding a grounding signal that purely visual metrics cannot offer. Using this pipeline, we evaluate 8 models spanning frontier closed-source generators, open-source generators, and robot-specific models. Our benchmark covers 101 manually curated manipulation tasks at three levels of physical complexity, measured across visual quality, trajectory fidelity, and execution success. Encouragingly, several models achieve measurable execution success, suggesting that generative priors learned from internet-scale data already encode meaningful physical knowledge. Yet visual quality proves a poor predictor of executability, exposing a dimension of model capability that standard visual evaluations do not capture. Dream.exe will be open-sourced at https://github.com/showlab/Dream.exe.