ACE-EGO-0:统一自我中心人类与机器人数据的VLA预训练框架
阅读原文· arxiv.org视觉-语言-动作(VLA)模型受限于机器人轨迹数据采集的高昂成本。ACE-EGO-0提出统一预训练框架,构建可扩展的第一人称视频到动作流水线,将原始人类视频转化为机器人格式伪动作轨迹。框架采用基于相机空间动作的统一表示、形态条件化和时间对齐动作分块,使伪标签与机器人演示可比。针对人类视频中的噪声伪动作,设计可靠性感知训练目标并引入人类辅助损失。模型在4.53K小时机器人/仿真数据和1.48K小时伪动作人类数据上训练,在RoboCasa GR1 TableTop和RoboTwin 2.0上达到最优,并展现对真实世界双臂操作的强迁移能力。
Vision-Language-Action (VLA) models benefit from large-scale and diverse embodied data, yet scaling robot trajectory collection is costly and labor-intensive. Recent advances show that large-scale egocentric human videos provide complementary real-world supervision in pretraining. However, joint training on human and robot data remains challenging due to divergences in action spaces, embodiment structures, temporal dynamics, and supervision quality. We introduce ACE-EGO-0, a unified VLA pretraining framework jointly leveraging heterogeneous data sources. To extract large-scale pretraining supervision from egocentric human videos, we build a scalable egocentric video-to-action pipeline that converts raw human videos into robot-format pseudo-action trajectories. To make these labels comparable with robot demonstrations, ACE-EGO-0 uses a unified action representation based on camera-space actions, morphology conditioning, and time-aligned action chunking. To robustly leverage noisy pseudo-action supervision from egocentric human videos, we formulate a reliability-aware training objective with a human auxiliary loss that concentrates supervision on reliable signals. We instantiate ACE-EGO-0 on 4.53K hours of robot and simulation data, together with 1.48K hours of pseudo-action-labeled egocentric human data. Experiments show that incorporating large-scale human supervision under reliability-aware weighting consistently improves both unified joint pretraining and supervised fine-tuning. ACE-EGO-0 achieves state-of-the-art performance on RoboCasa GR1 TableTop and RoboTwin 2.0, while demonstrating strong transfer to real-world bimanual manipulation.