学习长期运动嵌入以实现高效运动生成
阅读原文· arxiv.org研究团队提出了一种基于长期运动嵌入的高效运动生成方法,通过从大规模轨迹数据中学习高度压缩的运动表示,实现64倍时间压缩。该方法直接在运动潜空间上操作,而非合成完整视频,效率提升数个数量级。团队训练了条件流匹配模型,支持通过文本提示或空间戳记指定生成目标。实验表明,生成的运动分布在性能上超越了最先进的视频模型和专用任务方法,可生成长时间、真实的运动序列。
Understanding and predicting motion is a fundamental component of visual intelligence. Although modern video models exhibit strong comprehension of scene dynamics, exploring multiple possible futures through full video synthesis remains prohibitively inefficient. We model scene dynamics orders of magnitude more efficiently by directly operating on a long-term motion embedding that is learned from large-scale trajectories obtained from tracker models. This enables efficient generation of long, realistic motions that fulfill goals specified via text prompts or spatial pokes. To achieve this, we first learn a highly compressed motion embedding with a temporal compression factor of 64x. In this space, we train a conditional flow-matching model to generate motion latents conditioned on task descriptions. The resulting motion distributions outperform those of both state-of-the-art video models and specialized task-specific approaches.