HuggingFace Daily Papers(社区热门论文)
50AI 编辑部评分,满分 100

DreamX-Phi 1.0:面向机器人操作的动作条件视频世界模型

2026-08-13 08:00· 1天前
AI 导读

DreamX-Phi 1.0 是一个面向机器人操作的动作条件视频世界模型,输入观测帧、语言指令及动作序列即可预测未来观测。该模型通过 PRoPE 式几何编码注入每臂 SE(3) 变换,并引入深度分支与 SAM3 掩码保持物体一致性,同时用分布匹配蒸馏实现高效部署。

We present DreamX-Phi 1.0, an action-conditioned video world model for robotic manipulation that, given an observed frame, a language instruction, and a prescribed action sequence comprising end-effector poses and gripper states, predicts the resulting future observations. Yet realism alone does not guarantee faithfulness: a convincing rollout can still move the wrong arm or lose the manipulated object. To ensure the prediction respects each arm's commanded path, we inject per-arm SE(3) transformations into attention via PRoPE-style geometric encoding, preserving arm identity and rigid-motion structure. Action control alone does not fully constrain scene geometry or the evolution of small manipulated objects. We therefore add a lightweight depth branch for scene-level geometry and use SAM3 masks with a frozen V-JEPA teacher to maintain object consistency throughout grasping. We further distill the multi-step generator into a few-step student via distribution-matching distillation for efficient deployment. At the time of writing, achieves first place on Track1 and second place on Track2 of the WorldArena~2.0 Challenge. Our model and code will be publicly available.

来源:HuggingFace Daily Papers(社区热门论文) · arxiv.org

DreamX-Phi 1.0:面向机器人操作的动作条件视频世界模型

HuggingFace Daily Papers(社区热门论文)·2026-08-13 08:00·1天前
AI 导读

DreamX-Phi 1.0 是一个面向机器人操作的动作条件视频世界模型,输入观测帧、语言指令及动作序列即可预测未来观测。该模型通过 PRoPE 式几何编码注入每臂 SE(3) 变换,并引入深度分支与 SAM3 掩码保持物体一致性,同时用分布匹配蒸馏实现高效部署。

原文 · 保持原样,未翻译

We present DreamX-Phi 1.0, an action-conditioned video world model for robotic manipulation that, given an observed frame, a language instruction, and a prescribed action sequence comprising end-effector poses and gripper states, predicts the resulting future observations. Yet realism alone does not guarantee faithfulness: a convincing rollout can still move the wrong arm or lose the manipulated object. To ensure the prediction respects each arm's commanded path, we inject per-arm SE(3) transformations into attention via PRoPE-style geometric encoding, preserving arm identity and rigid-motion structure. Action control alone does not fully constrain scene geometry or the evolution of small manipulated objects. We therefore add a lightweight depth branch for scene-level geometry and use SAM3 masks with a frozen V-JEPA teacher to maintain object consistency throughout grasping. We further distill the multi-step generator into a few-step student via distribution-matching distillation for efficient deployment. At the time of writing, achieves first place on Track1 and second place on Track2 of the WorldArena~2.0 Challenge. Our model and code will be publicly available.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org