DrawMotion:通过手绘生成3D人体动作
阅读原文· arxiv.org该研究提出了DrawMotion,一个基于扩散的多条件动作生成框架,旨在解决传统文本到动作生成中用户意图表达不精确的问题。该框架支持文本与手绘两种控制条件,分别提供语义和空间引导。技术上,它通过算法自动生成手绘火柴人草图、设计多条件融合模块以降低计算复杂度,并利用无训练引导方法将生成动作与用户意图对齐。实验表明,手绘方式能将用户生成符合想象的动作所需时间缩短约46.7%。相关代码与演示已开源。
Text-to-motion generation, which translates textual descriptions into human motions, faces the challenge that users often struggle to precisely convey their intended motions through text alone. To address this issue, this paper introduces DrawMotion, an efficient diffusion-based framework designed for multi-condition scenarios. DrawMotion generates motions based on both a conventional text condition and a novel hand-drawing condition, which provide semantic and spatial control over the generated motions, respectively. Specifically, we tackle the fine-grained motion generation task from three perspectives: 1) freehand drawing condition. To accurately capture users' intended motions without requiring tedious textual input, we develop an algorithm to automatically generate hand-drawn stickman sketches across different dataset formats; 2) multi-condition fusion. We propose a Multi-Condition Module (MCM) that is integrated into the diffusion process, enabling the model to exploit all possible condition combinations while reducing computational complexity compared to conventional approaches; and 3) training-free guidance. Notably, the MCM in DrawMotion ensures that its intermediate features lie in a continuous space, allowing classifier-guidance gradients to update the features and thereby aligning the generated motions with user intentions while preserving fidelity. Quantitative experiments and user studies demonstrate that the freehand drawing approach reduces user time by approximately 46.7% when generating motions aligned with their imagination. The code, demos, and relevant data are publicly available at https://github.com/InvertedForest/DrawMotion.