ActiveMimic:基于主动感知的自我中心视频预训练
阅读原文· arxiv.orgActiveMimic 是一个预训练框架,从单个人体佩戴的 RGB 摄像头中恢复同步的相机和手腕轨迹,将相机运动建模为视角动作,从而在野外第一人称人类视频上联合学习主动感知和操作能力,再迁移至目标机器人。跨任务真实世界实验表明,ActiveMimic 持续超越基于人类视频预训练的基线,并达到与基于机器人数据预训练的 SOTA 模型相当的性能。进一步分析确认主动感知能力源自第一人称视频预训练,而非机器人微调。
Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.