平移作为桥接动作:从人类到机器人的操作技能迁移
阅读原文· arxiv.org研究从人类动作数据向双臂平行夹爪机器人转移操作技能的方法。针对6DoF人体姿态估计噪声大、接触模式差异大的问题,提出桥接动作表示——初始头部相机帧内的相对手腕平移,作为人类与机器人共有的动作空间。构建π_0-like视觉-语言-动作模型,通过交错动作token和注意力掩码处理不同形态间动作分量的缺失。在双臂操作任务上,该方法比噪声6DoF动作更有效地转移人类操作知识,且效果随人类数据量增加而提升。
We study whether we can learn novel manipulation skills from human actions to a bi-manual robot with parallel grippers. Human action data is cheap, abundant, and diverse, making it one of the most promising resources for scaling up robot learning. Yet transferring skills from humans to robots remains hard: most prior work treats humans as just another bi-manual 6DoF embodiment, where hand-pose estimates are noisy and the contact patterns of human fingers differ fundamentally from those of a parallel gripper. We argue that learning rotation-inclusive action signals from human data is therefore sub-optimal, and instead propose a bridging action representation: the relative wrist translation within the initial head-camera frame, an action space shared by humans and robots. To handle the potential absence of certain action components in different embodiments, we build a π_0-like vision-language-action model with interleaved action tokens and attention masking. On a suite of novel bi-manual manipulation tasks, our bridging action transfers human manipulation knowledge to robots far more effectively than noisy 6DoF human actions and scales with the amount of human data.