Zhixue Fang

, Zhimin Zhang

, Bi’an Du

, Zijie Meng

, Yan Zhou

, Wei Hu

, Guoxin Zhang

Abstract

Video motion transfer aims to animate a target object using dynamics from a reference video. Existing formulations largely rely on fixed structural correspondence, which becomes ill-defined when reference and target objects differ substantially in morphology, articulation, or deformation mechanisms. We introduce Motion Beyond Morphology, a perspective that seeks to transfer motion beyond fixed structural correspondence, by preserving dynamics that remain meaningful across different target morphologies. To realize this, we propose a two-stage framework. Stage I learns complementary multi-granularity abstract motion views and uses them to bootstrap cross-category video pairs that preserve transferable dynamics across diverse morphologies. Stage II internalizes this supervision into direct reference-video-conditioned generation, removing the need for explicit motion extraction at inference. We further introduce OpenVMT-Dataset and OpenVMT-Bench for training and evaluating image- and text-conditioned motion transfer across Same, Near, and Far category gaps, and plan to release both upon acceptance. Extensive experiments demonstrate state-of-the-art motion fidelity and target preservation. Project page: https://miniz233.github.io/MotionBeyondMorphology/

Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations

Zhixue Fang^1,*, Zhimin Zhang^2,1,*, Bi’an Du^2,1, Zijie Meng^2,1, Yan Zhou^1,, Wei Hu^2,, Guoxin Zhang¹, Pengfei Wan¹, Kun Gai¹

¹Kling Team, Kuaishou Technology

²Peking University

媒体内容 · 前往原文查看

Introduction

Video motion transfer aims to animate a target object according to the dynamics of a source video while preserving the appearance and scene specified by the target condition. Existing approaches are predominantly developed for source-target pairs with compatible structural templates. Human and character animation, for instance, commonly relies on pose or skeleton sequences, where source and target share comparable articulated structures and kinematic layouts (Xu et al. 2024; Hu et al. 2024). Beyond human-centric animation, existing methods often extend to other domains by adopting representations tailored to specific structural patterns or motion regimes. While effective within their intended settings, such formulations remain difficult to generalize when source and target exhibit substantially different morphologies or motion mechanisms.

Recent studies broaden the scope of motion transfer to reduce such category-specific assumptions through implicit and explicit motion representations. Implicit features derived from intermediate activations or attention patterns encode rich spatiotemporal dynamics and offer broad category applicability (Ling et al. 2025; Pondaven et al. 2025; Shi et al. 2025), yet may retain source-specific appearance, geometry, or background information. Explicit representations provide stronger separation from visual content, but typically capture only selected aspects of motion. Object trajectories capture object-level motion, such as global translation and displacement (Wang et al. 2024; Wu et al. 2024b), but provide limited information about internal articulation and deformation. Dense point tracks offer finer-grained local motion cues by tracing point-wise dynamics, yet lack explicit semantic, pose, and topological structure (Geng et al. 2025). Consequently, different representations offer complementary strengths, and no single representation remains equally expressive and reliable across diverse object categories and motion patterns.

These limitations motivate a broader formulation of open-category motion transfer, where source and target may differ substantially in semantics, morphology, and motion topology. We argue that motion transfer across open categories should not rely on a fixed notion of correspondence. Instead, the model should adaptively determine which motion factors should remain coupled between source and target and which should be disentangled according to their structural compatibility. For closely related categories, semantic correspondence, pose, and topology may provide meaningful constraints together with trajectory and temporal dynamics. As the morphological gap increases, transferable information may shift toward more abstract factors such as trajectory, rhythm, and local deformation, while source-specific semantics and structure should be progressively weakened. The relevant combination can vary across individual source–target pairs, making adaptive coupling a key requirement for general motion transfer. We refer to this perspective as Motion Beyond Morphology, where motion is transferred through the factors that remain meaningful under the target morphology rather than through a fixed structural template. Figure 1 illustrates such transfers across large morphological gaps.

Learning this adaptive correspondence presents a distinct supervision challenge. Raw videos jointly encode motion, appearance, geometry, and spatial layout, while large-scale video collections rarely provide paired examples that reveal which dynamics should remain invariant across different morphologies. We therefore use motion abstractions not as a fixed representation for final inference, but as complementary mechanisms for constructing the cross-morphology supervision required to learn such correspondence.

Based on this insight, we propose a two-stage framework. In Stage I, Abstract Motion Bootstrapping, we establish a shared conditioning interface for complementary motion views, including semantic kinematics, global trajectories, dense point tracks, structural edges and 6-DoF axis. Each view captures a different aspect of motion and is applied only when its extraction is reliable. By pairing an extracted motion condition with target conditions from different categories, Stage I synthesizes videos that differ in appearance, morphology, and background while preserving the motion attributes specified by the condition. These samples form cross-category motion-related pairs and provide explicit supervision for motion correspondence beyond a fixed structural template.

The explicit abstractions in Stage I provide controllable supervision, but their individual coverage and information capacity remain inherently limited. Our final goal is instead to infer transferable dynamics directly from a raw source video. In Stage II, Cross-Category Motion Internalization, we initialize from Stage I and replace the explicit motion condition with the source video itself. Training on cross-category pairs makes source-specific appearance and morphology unreliable predictors of the target, encouraging the model to identify dynamics that remain informative across category changes and reconstruct their realization according to the target condition. At inference, the model directly takes a source video together with either a reference image or text prompt, without requiring skeleton extraction, trajectory specification, point tracking, or per-video optimization.

We further introduce OpenVMT-Dataset and OpenVMT-Bench for training and systematic evaluation. To our knowledge, OpenVMT-Dataset is the first dedicated training dataset of open-category, cross-content video pairs with instance-level motion equivalence. It contains 10K motion-equivalent pairs spanning diverse morphologies and motion patterns, providing direct supervision for cross-category motion correspondence. OpenVMT-Bench evaluates motion transfer under progressively increasing category gaps through Same, Near, and Far splits. Our method achieves state-of-the-art performance on the proposed benchmark, demonstrating robust motion fidelity and target preservation across diverse source–target relations.

• •

  We formulate open-category motion transfer from the perspective of Motion Beyond Morphology, where transferable motion attributes are adaptively preserved according to the structural relationship between source and target.

• •

  We propose a two-stage framework that leverages complementary motion abstractions to bootstrap cross-category supervision and subsequently internalizes this supervision into direct source-video-conditioned motion transfer.

• •

  We introduce OpenVMT-Dataset, the first open-category training dataset with instance-level motion-equivalent cross-content pairs, and OpenVMT-Bench, on which our method achieves state-of-the-art performance.

Related Work

Structure-conditioned animation.

Most motion-transfer methods have been developed for humans, portraits, and articulated characters. Pose-guided animation methods (Ma et al. 2024; Hu et al. 2024; Xu et al. 2024; Fang et al. 2026; Team et al. 2026) obtain precise control from keypoints, skeletons, or dense pose. Their structural assumptions are useful within a category but do not extend naturally to arbitrary objects. The same limitation applies to landmark-based face and body animation, where the motion signal is defined on a fixed morphology.

Reference-based motion transfer.

Recent diffusion-based methods reduce reliance on category-specific structures by extracting motion directly from reference videos (Gao et al. 2025; Ling et al. 2025; Yesiltepe et al. 2024; Pondaven et al. 2025; Ressler-Antal et al. 2025). MotionClone (Ling et al. 2025), DiTFlow (Pondaven et al. 2025), and MotionShop (Yesiltepe et al. 2024) leverage temporal attention, motion flow, or diffusion guidance for motion transfer, while DeT (Shi et al. 2025) and DisMo (Ressler-Antal et al. 2025) further explore motion–appearance decoupling and transferable motion representations. Despite broader applicability, such motion cues may still retain source-specific shape, layout, or appearance. Our method instead uses complementary explicit motion abstractions to construct cross-category supervision, which is subsequently internalized into direct source-video-conditioned transfer.

Motion representations and evaluation.

Trajectory-based methods (Yin et al. 2023; Wang et al. 2024; Shi et al. 2024) control coarse motion, track-based methods (Zhou et al. 2025) capture finer local dynamics, and pose-based animation (Hu et al. 2024; Xu et al. 2024) provides semantic structure for articulated subjects. Hybrid approaches (Wang et al. 2023) combine multiple cues. We instead use complementary motion abstractions according to their applicability across diverse objects.

General benchmarks such as VBench (Huang et al. 2024a) and VBench++ (Huang et al. 2024b) evaluate overall video quality, while MotionBench (Yesiltepe et al. 2024) and MTBench (Shi et al. 2025) target motion transfer. Existing benchmarks offer limited control over structural gaps and rarely include curated target images. OpenVMT-Bench supports T2V and I2V with Same, Near, and Far splits for motion fidelity, target preservation, and source leakage.

Method

Overview

Given a reference video and a target condition , our goal is to synthesize

(1)

such that follows the transferable dynamics of , while its appearance, morphology, and scene content are determined by . Here, comprises a target text description and an optional reference image, covering both T2V and I2V settings. Our framework follows a two-stage training scheme. Stage I learns motion-conditioned generation from multi-granularity abstract motion views and uses the learned model to bootstrap cross-category video pairs with shared dynamics but different appearances and morphologies. Since these abstract views are representation-specific and require explicit extraction, Stage I serves primarily to construct cross-category supervision rather than as the final transfer model. Stage II initializes from Stage I and replaces the abstract condition with the raw reference video, thereby internalizing transferable motion into direct video-conditioned generation. At inference, motion transfer is performed directly from the reference video under text and optional image conditioning, without explicit motion extraction or test-time optimization.

Multi-Granularity Abstract Motion Views

No single representation can reliably cover the diverse objects and motion patterns in open-category motion transfer. We therefore employ multi-granularity abstract motion views, each capturing complementary motion attributes and applied only to the data for which it can be reliably extracted.

Semantic kinematics.

We use ViTPose++ (Xu et al. 2023) to extract animal keypoints and render skeleton sequences, providing semantically meaningful control for articulated motions such as limb movement and wing articulation.

Depth-aware global trajectories.

For object-level motion, we track a stable object center derived from foreground masks and render a depth-aware circle sequence using the tracked position and mean foreground depth. This representation captures global displacement together with approaching and receding dynamics while largely suppressing object-specific structure.

Dense point tracks.

We extract dense trajectories with TAPNext++ (Jung et al. 2026), which capture local motion and deformation without predefined structural semantics, making them applicable to diverse non-rigid objects.

6-DoF axis.

For rigid or approximately rigid objects, we estimate frame-wise rigid transformations from the 3D tracks produced by SpaTrackerV2 (Xiao et al. 2025) and render them as coordinate-axis sequences, explicitly representing turning and orientation changes.

Edges.

We further use foreground Canny edges to preserve coarse shape evolution while suppressing most appearance and background information, providing a broadly applicable structural motion cue.

These motion views provide complementary supervision across diverse motion regimes. View applicability is defined over 100+ object categories and motion patterns, with one applicable view sampled per video. After view-specific perturbations, all views are unified into a video-like form, yielding . Detailed mappings are provided in the supplementary material.

Stage I: Abstract Motion Bootstrapping

Abstract motion grounding.

Stage I grounds heterogeneous motion abstractions into the generative model, enabling them to serve as controllable motion conditions. For each training tuple , consists of the first frame of and its VLM-generated caption. After representation-specific augmentation, the motion view and the first-frame image are encoded by the pretrained 3D VAE, while the caption is processed by the text encoder. These conditions are incorporated with the noisy target latent through the backbone’s native conditioning interface, and the model is trained to reconstruct . This formulation allows a single generator to learn heterogeneous motion abstractions without representation-specific control branches.

Following the flow-matching objective of the base model, we optimize

(2)

where , and and denote the noisy latent of and the corresponding target velocity, respectively.

Cross-category motion-pair generation.

Once grounded, the abstract views provide an intermediate motion interface for synthesizing dynamics across different visual contents. Given a reference video , we extract an applicable motion view

(3)

and pair it with a target condition selected from a different category. The Stage-I model then generates

(4)

This yields candidate triplets , where the reference and target differ in appearance and morphology while sharing the motion attributes conveyed by . In this way, Stage I converts readily extractable motion abstractions into cross-category supervision that is difficult to obtain from real videos directly.

Cross-category pair filtering.

To ensure the reliability of the bootstrapped supervision, we perform quality control both before and after synthesis. We first reject samples with unreliable motion extraction using representation-specific validity checks. For each generated triplet , we further assess four complementary criteria: motion fidelity, target fidelity, reference leakage, and video quality. These criteria respectively evaluate whether preserves the intended dynamics, follows the target condition, avoids inheriting reference-specific appearance or structure, and maintains satisfactory visual and temporal quality. Please refer to the supplementary materials for details. Only candidates satisfying all criteria are retained, forming the filtered bootstrapped dataset used in Stage II.

媒体内容 · 前往原文查看

Method	Motion Fidelity					Target Fidelity	Leakage	Temporal
	HMF-S	HMF-N	HMF-F	HMF-All	G-Mot.	Align.	G-Leak	Smooth.
OpenVMT-I2V  Align.: DINO-I
DisMo	0.7367	0.7437	0.7441	0.7412	2.811	0.8587	1.182	0.8018
Wan-Move	0.7370	0.7338	0.7335	0.7349	2.019	0.8849	1.128	0.9498
Tora	0.7161	0.7259	0.7402	0.7262	2.377	0.8846	1.023	0.9310
Ours	0.7435	0.7617	0.7446	0.7505	3.575	0.9058	1.000	0.9651
OpenVMT-T2V  Align.: CLIP-T
DeT	0.7441	0.7486	0.7358	0.7428	2.418	0.2392	1.152	0.9031
FlowMotion	0.7384	0.7440	0.7337	0.7387	2.970	0.2522	1.170	0.8859
DisMo-T2V	0.7680	0.7666	0.7742	0.7696	2.873	0.2415	1.109	0.9265
Ours	0.7722	0.7712	0.7688	0.7707	3.976	0.2677	1.067	0.9122

Stage II: Cross-Category Motion Internalization

Stage I provides effective cross-category supervision through abstract motion views, but these representations are inherently representation-specific: they require dedicated extractors and retain only selected aspects of the underlying motion. Our goal in Stage II is therefore to internalize this supervision into the generator, such that transferable dynamics can be inferred directly from the raw reference video without committing to a predefined motion abstraction.

We initialize the model from Stage I, , and replace the abstract motion condition with the reference video . Using the same video-like conditioning interface, the reference video is encoded as

(5)

and conditions the generation of the bootstrapped target together with . For each , we optimize

(6)

where and denote the noisy latent of and its corresponding target velocity.

The cross-category construction of is essential to this internalization process. Although and differ substantially in appearance and morphology, they preserve the motion attributes transferred through Stage I. Consequently, reference-specific appearance and structure no longer consistently predict the target video, whereas the shared temporal dynamics remain informative across category changes. Training on such pairs therefore discourages reliance on morphology-specific shortcuts and encourages the model to extract transferable motion directly from RGB observations and realize it according to the target condition.

Inference

At inference, motion transfer is performed directly from the reference video,

(7)

where denotes the target text condition with an optional reference image, covering both T2V and I2V settings. The reference video provides the motion signal, whereas determines the target appearance, morphology, and scene content. Starting from a Gaussian noise latent, the model progressively generates the target video through iterative denoising conditioned on both and .

The resulting inference pipeline eliminates the need for explicit motion extraction and per-video optimization, enabling unified motion transfer across diverse target categories within a single model.

Experiments

Experimental Setup

Dataset and benchmark.

We train our model on a large-scale collection of approximately 1.6M object-centric motion videos curated from the Internet, covering diverse subjects and motion patterns. In addition, we curate OpenVMT-Dataset, a release set of 10K high-quality cross-content motion-equivalent video pairs derived from our bootstrapped data, comprising 4K Same, 4K Near, and 2K Far pairs. Videos within each pair exhibit equivalent motion dynamics while differing in subject identity, morphology, and scene background. Such paired data provides explicit supervision for learning motion beyond appearance and structural correspondence, complementing the diverse but unpaired Internet videos.

For evaluation, we introduce OpenVMT-Bench, consisting of an image-conditioned track, OpenVMT-I2V, and a text-conditioned track, OpenVMT-T2V. Each track is divided into Same, Near, and Far splits with progressively larger semantic and morphological gaps between the source motion and target content: Same pairs share closely matched categories and structures, Near pairs exhibit moderate cross-category variation, and Far pairs involve substantially different subjects with weak or absent part-level correspondence. OpenVMT-I2V and OpenVMT-T2V contain 123 and 166 test cases, respectively. Detailed dataset and benchmark statistics are provided in the supplementary material.

Baselines.

We select representative open-source motion-transfer methods spanning different conditioning paradigms. For OpenVMT-I2V, we compare against DisMo (Ressler-Antal et al. 2025), which uses an implicit motion representation, and the trajectory-conditioned methods Wan-Move (Chu et al. 2025) and Tora (Zhang et al. 2025). For trajectory-based baselines, we extract foreground point trajectories using CoTracker2 (Karaev et al. 2024) as motion conditions. For OpenVMT-T2V, we compare against DeT (Shi et al. 2025), FlowMotion (Wang et al. 2026), and a text-conditioned adaptation of DisMo (DisMo-T2V) (Ressler-Antal et al. 2025), obtained by disabling its image condition while retaining text and motion conditioning. We use official implementations and released checkpoints whenever available, and align output duration, resolution, and frame rate across methods. We additionally provide qualitative comparisons with commercial systems, including Seedance-2.0 (Seedance et al. 2026), Kling Motion Brush (Kuaishou Technology 2026), and Kling Omni3 (Team et al. 2025).

Metrics.

We evaluate motion transfer along four dimensions: motion fidelity, target fidelity, source leakage, and temporal quality. Motion fidelity is measured by Hybrid Motion Fidelity (HMF) (Shi et al. 2025), which captures consistency in global trajectories and local motion dynamics, together with a Gemini-based motion preservation score (G-Mot.) that assesses perceptual correspondence between the source and generated motions. Target fidelity is evaluated by DINO-I (Oquab et al. 2024) for OpenVMT-I2V, measuring visual similarity to the reference image, and by CLIP-T (Wu et al. 2024a) for OpenVMT-T2V, measuring alignment with the target text prompt. Source leakage is assessed using a Gemini-based leakage score (G-Leak), which measures the presence of source-specific appearance, structure, or background content in the generated video, with lower values indicating less leakage. Temporal quality is evaluated using the motion smoothness metric from VBench (Huang et al. 2024a). Since automatic metrics may not fully capture perceptual differences in motion transfer quality, we additionally conduct a human Good/Same/Bad (GSB) preference study. Detailed metric definitions and evaluation protocols are provided in the supplementary material.

Implementation details.

Our model is initialized from a MMDiT-based image-to-video diffusion backbone with a causal 3D VAE. Following the backbone training configuration, we use 121-frame video clips resized to a target area of pixels while preserving the original aspect ratio. We optimize the model using Adam with a global batch size of 128. Stage I is trained for 15K iterations on the large-scale unpaired motion video corpus. Stage II is initialized from the Stage-I model and further trained for 4K iterations on the constructed motion-equivalent pairs.

Comparison with State-of-the-Art Methods

Image-to-video motion transfer.

Table 1 reports results on OpenVMT-I2V. Our method achieves the best performance across all reported metrics. It consistently obtains the highest HMF from Same to Far transfers, together with a substantially higher G-Mot., demonstrating robust motion fidelity across category gaps. It also achieves the best target fidelity and lowest G-Leak, indicating stronger target preservation with less source-content contamination. The highest smoothness score further confirms that these gains are achieved without sacrificing temporal quality.

Text-to-video motion transfer.

Table 1 reports results on OpenVMT-T2V. Our method achieves the best overall HMF, G-Mot., CLIP-T, and G-Leak, demonstrating a strong balance between motion preservation and text-conditioned target generation. While DisMo-T2V slightly outperforms ours in HMF-F and temporal smoothness, its stronger adherence to source motion is accompanied by lower target alignment and higher source leakage, indicating a tendency to preserve source-specific motion structures at the expense of faithful target realization, as shown in Fig. 3. In contrast, our method substantially improves perceptual motion fidelity and text alignment while maintaining competitive motion consistency, enabling more reliable motion transfer across category gaps.

Perceptual evaluation.

We further conduct a human Good/Same/Bad (GSB) preference study with 12 evaluators experienced in video generation. Since GSB evaluation is pairwise, we report results against the strongest representative baseline for each benchmark track, with DisMo used for OpenVMT-I2V and FlowMotion for OpenVMT-T2V. We report both preference rate, , and decisive win rate, . As shown in Table 2, our method is consistently preferred in motion fidelity, target preservation, and overall quality, achieving overall preference rates of 93.0% on I2V and 97.3% on T2V.

媒体内容 · 前往原文查看

Task	Criterion	G	S	B	Pref.	Win
I2V	Motion	82.8	13.1	4.1	89.3	95.3
	Target	67.2	24.6	8.2	79.5	89.1
	Overall	88.5	9.0	2.5	93.0	97.3
T2V	Motion	82.9	15.9	1.2	90.9	98.6
	Target	77.4	20.7	1.8	87.8	97.7
	Overall	95.7	3.0	1.2	97.3	98.7

Qualitative Comparison

Figure 3 presents qualitative comparisons on I2V (top), T2V (middle), and commercial video generators (bottom). For I2V, trajectory-based methods mainly capture coarse displacement, while DisMo may retain source-specific structures under large morphology gaps. Our method better preserves fine-grained motion while maintaining the target appearance and structure. For T2V, it more faithfully transfers both global and local dynamics while preserving the target semantics. Compared with commercial models, including Seedance-2.0 (Seedance et al. 2026), Kling Motion Brush (Kuaishou Technology 2026), and Kling Omni3 (Team et al. 2025), our method shows stronger reference-motion adherence across large morphology gaps, faithfully reproducing the specific dynamics of the source video rather than merely plausible target motion.

媒体内容 · 前往原文查看

Variant	HMF-All	HMF-F	G-Mot.
Stage I: Multi-granularity motion views
Trajectory only	0.7524	0.7582	3.354
Track only	0.7593	0.7575	3.673
Edge only	0.7656	0.7638	3.646
Stage II: Cross-category bootstrapping
In-category bootstrap	0.7588	0.7538	3.436
Full model	0.7707	0.7688	3.976

Ablation studies

Table 3 reports quantitative ablations on OpenVMT-T2V, while Fig. 4 shows qualitative results on OpenVMT-I2V. Replacing cross-category pairs in Stage II with in-category pairs reduces HMF-All from 0.7707 to 0.7588 and G-Mot. from 3.976 to 3.436, confirming the benefit of morphology-diverse supervision.

We further analyze individual motion views with Stage I to characterize their representational behavior. Trajectories favor global displacement, tracks capture local dynamics, and edges provide stronger structural cues, highlighting their complementary roles across motion regimes. We focus on broadly applicable views here, with additional ablations provided in the supplementary material.

Conclusion

We presented Motion Beyond Morphology, a two-stage framework for open-category video motion transfer. The central idea is to use heterogeneous motion abstractions to construct cross-category supervision rather than imposing a fixed representation at inference. Stage I grounds semantic kinematics, depth-aware global trajectories, dense point tracks, 6-DoF axis, and structural edges through a shared conditioning interface, and uses them to bootstrap motion-equivalent videos across diverse morphologies. Stage II internalizes this supervision into direct source-video-conditioned generation, supporting both image- and text-conditioned motion transfer without explicit motion extraction at inference. Experiments on OpenVMT-Bench examine motion fidelity, target preservation, and source-content leakage across increasingly difficult category gaps.

References

• R. Chu, Y. He, Z. Chen, S. Zhang, X. Xu, b. xia, D. WANG, H. Yi, X. Liu, H. Zhao, Y. Liu, Y. Zhang, and Y. Yang (2025) Wan-move: motion-controllable video generation via latent trajectory guidance. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, pp. 404–432. External Links: Link Cited by: Baselines..
• Z. Fang, X. He, S. Tang, H. Zhang, Q. Li, X. Liu, P. Wan, and K. Gai (2026) 3D-aware implicit motion control for view-adaptive human video generation. External Links: 2602.03796, Link Cited by: Structure-conditioned animation..
• J. Gao, Z. Yin, C. Hua, Y. Peng, K. Liang, Z. Ma, J. Guo, and Y. Liu (2025) Conmo: controllable motion disentanglement and recomposition for zero-shot motion transfer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7191–7200. Cited by: Reference-based motion transfer..
• D. Geng, C. Herrmann, J. Hur, F. Cole, S. Zhang, T. Pfaff, T. Lopez-Guevara, Y. Aytar, M. Rubinstein, C. Sun, et al. (2025) Motion prompting: controlling video generation with motion trajectories. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1–12. Cited by: Introduction.
• L. Hu, X. Gao, P. Zhang, K. Sun, B. Zhang, and L. Bo (2024) Animate anyone: consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Introduction, Structure-conditioned animation., Motion representations and evaluation..
• Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y. Wang, X. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu (2024a) VBench: comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Motion representations and evaluation., Metrics..
• Z. Huang, F. Zhang, X. Xu, Y. He, J. Yu, Z. Dong, Q. Ma, N. Chanpaisit, C. Si, Y. Jiang, Y. Wang, X. Chen, Y. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu (2024b) VBench++: comprehensive and versatile benchmark suite for video generative models. arXiv preprint arXiv:2411.13503. Cited by: Motion representations and evaluation..
• S. Jung, A. Zholus, M. Sundermeyer, C. Doersch, R. Goroshin, D. J. Tan, S. Chandar, R. Triebel, and F. Tombari (2026) TAPNext++: what’s next for tracking any point (tap)?. arXiv preprint arXiv:2604.10582. Cited by: Dense point tracks..
• N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rupprecht (2024) Cotracker: it is better to track together. In European conference on computer vision, pp. 18–35. Cited by: Baselines..
• Kuaishou Technology (2026) Kling ai. Note: https://kling-ai.prt.kuaishou.com/appAccessed: 2026-07-29 Cited by: Baselines., Qualitative Comparison.
• P. Ling, J. Bu, P. Zhang, X. Dong, Y. Zang, T. Wu, H. Chen, J. Wang, and Y. Jin (2025) MotionClone: training-free motion cloning for controllable video generation. In International Conference on Learning Representations, Cited by: Introduction, Reference-based motion transfer..
• Y. Ma, Y. He, X. Cun, X. Wang, S. Chen, Y. Shan, X. Li, and Q. Chen (2024) Follow your pose: pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, Cited by: Structure-conditioned animation..
• M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. (2024) Dinov2: learning robust visual features without supervision. Transactions on Machine Learning Research Journal. Cited by: Metrics..
• A. Pondaven, A. Siarohin, S. Tulyakov, P. Torr, and F. Pizzati (2025) Video motion transfer with diffusion transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Introduction, Reference-based motion transfer..
• T. Ressler-Antal, F. Fundel, M. B. Alaya, S. A. Baumann, F. Krause, M. Gui, and B. Ommer (2025) DisMo: disentangled motion representations for open-world motion transfer. In Advances in Neural Information Processing Systems, Cited by: Reference-based motion transfer., Baselines..
• T. Seedance, D. Chen, L. Chen, X. Chen, Y. Chen, Z. Chen, Z. Chen, F. Cheng, T. Cheng, Y. Cheng, et al. (2026) Seedance 2.0: advancing video generation for world complexity. arXiv preprint arXiv:2604.14148. Cited by: Baselines., Qualitative Comparison.
• Q. Shi, J. Wu, J. Bai, J. Zhang, L. Qi, X. Li, and Y. Tong (2025) Decouple and track: benchmarking and improving video diffusion transformers for motion transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: Introduction, Reference-based motion transfer., Motion representations and evaluation., Baselines., Metrics..
• X. Shi, Z. Huang, F. Wang, W. Bian, D. Li, Y. Zhang, M. Zhang, K. C. Cheung, S. See, H. Qin, et al. (2024) Motion-i2v: consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH 2024 Conference Papers, pp. 1–11. Cited by: Motion representations and evaluation..
• K. Team, J. Chen, Y. Ci, X. Du, Z. Feng, K. Gai, S. Guo, F. Han, J. He, K. He, et al. (2025) Kling-omni technical report. arXiv preprint arXiv:2512.16776. Cited by: Baselines., Qualitative Comparison.
• K. Team, J. Chen, Y. Ding, Z. Fang, K. Gai, K. He, X. He, J. Hua, M. Lao, X. Li, et al. (2026) Kling-motioncontrol technical report. arXiv preprint arXiv:2603.03160. Cited by: Structure-conditioned animation..
• X. Wang, H. Yuan, S. Zhang, D. Chen, J. Wang, Y. Zhang, Y. Shen, D. Zhao, and J. Zhou (2023) VideoComposer: compositional video synthesis with motion controllability. In Advances in Neural Information Processing Systems, Cited by: Motion representations and evaluation..
• Z. Wang, Y. Xu, J. Xiao, and L. Chen (2026) FlowMotion: training-free flow guidance for video motion transfer. arXiv preprint arXiv:2603.06289. Cited by: Baselines..
• Z. Wang, Z. Yuan, X. Wang, Y. Li, T. Chen, M. Xia, P. Luo, and Y. Shan (2024) Motionctrl: a unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1–11. Cited by: Introduction, Motion representations and evaluation..
• J. Wu, X. Li, Y. Zeng, J. Zhang, Q. Zhou, Y. Li, Y. Tong, and K. Chen (2024a) Motionbooth: motion-aware customized text-to-video generation. Advances in Neural Information Processing Systems 37, pp. 34322–34348. Cited by: Metrics..
• W. Wu, Z. Li, Y. Gu, R. Zhao, Y. He, D. J. Zhang, M. Z. Shou, Y. Li, T. Gao, and D. Zhang (2024b) Draganything: motion control for anything using entity representation. In European Conference on Computer Vision, pp. 331–348. Cited by: Introduction.
• Y. Xiao, J. Wang, N. Xue, N. Karaev, I. Makarov, B. Kang, X. Zhu, H. Bao, Y. Shen, and X. Zhou (2025) Spatialtrackerv2: 3d point tracking made easy. In ICCV 2025 Workshop on Wild 3D: 3D Modeling, Reconstruction, and Generation in the Wild, Cited by: 6-DoF axis..
• Y. Xu, J. Zhang, Q. Zhang, and D. Tao (2023) Vitpose++: vision transformer for generic body pose estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (2), pp. 1212–1230. Cited by: Semantic kinematics..
• Z. Xu, J. Zhang, J. H. Liew, H. Yan, J. Liu, C. Zhang, J. Feng, and M. Z. Shou (2024) MagicAnimate: temporally consistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Introduction, Structure-conditioned animation., Motion representations and evaluation..
• H. Yesiltepe, T. H. S. Meral, C. Dunlop, and P. Yanardag (2024) MotionShop: zero-shot motion transfer in video diffusion models with mixture of score guidance. arXiv preprint arXiv:2412.05355. Cited by: Reference-based motion transfer., Motion representations and evaluation..
• S. Yin, C. Wu, J. Liang, J. Shi, H. Li, G. Ming, and N. Duan (2023) DragNUWA: fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089. Cited by: Motion representations and evaluation..
• Z. Zhang, J. Liao, M. Li, Z. Dai, B. Qiu, S. Zhu, L. Qin, and W. Wang (2025) Tora: trajectory-oriented diffusion transformer for video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2063–2073. Cited by: Baselines..
• H. Zhou, C. Wang, R. Nie, J. Liu, D. Yu, Q. Yu, and C. Wang (2025) Trackgo: a flexible and efficient method for controllable video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, pp. 10743–10751. Cited by: Motion representations and evaluation..