Yuancheng Xu

Netflix

United States of America

and Eyeline Labs

United States of America

xuyuancheng0@gmail.com

Mingming He

United States of America

hmm.lillian@gmail.com

Pablo Salamanca

Netflix

United States of America

and Eyeline Labs

United States of America

Pablosalamanca88@gmail.com

Li Ma

Eyeline Labs

United States of America

li.ma@scanlinevfx.com

Yash Kant

Netflix

United States of America

and Eyeline Labs

United States of America

ysh.kant@gmail.com

Emmett Steven

Netflix

United States of America

esteven@netflix.com

Paul Debevec

Netflix

United States of America

and Eyeline Labs

United States of America

debevec@netflix.com

Ning Yu

Netflix

United States of America

and Eyeline Labs

United States of America

ningyu.hust@gmail.com

Abstract.

In visual storytelling, human performances are central to creative intent and narrative meaning. However, preserving human identity and performance while enabling flexible visual edits remains a fundamental challenge for generative video models. We formalize this challenge as identity-preserving video restylization, a task that propagates scene, lighting, and style changes specified by an edited keyframe across a source video, while strictly preserving facial likeness and performance, including expressions, eye gaze, and lip synchronization. A key obstacle is the absence of paired training data, as identity-preserving restylized video pairs are rarely available in real-world settings. To address this, we propose a principled decoupling between source-grounded identity preservation and edit-driven video synthesis. Our key insight is that facial appearance and expression should remain invariant, with illumination being the primary permissible variation. We therefore cast identity preservation as a video relighting problem, while modeling visual edit propagation as controlled video synthesis guided by the edited keyframe. Building on this formulation, we introduce ID-V2V, a video-to-video generative framework that integrates complementary control signals: relit facial regions and facial normal maps tightly constrain facial likeness and performance, while edited keyframes and depth sequences enable flexible and temporally coherent generation. This design enables effective construction of training pairs from a single video, eliminating the need for scarce paired data. Extensive experiments demonstrate that ID-V2V significantly outperforms existing methods in preserving facial likeness and fine-grained facial performance, robustly supports both single- and multi-subject scenarios, and delivers high visual quality, highlighting its strong potential as a human-centric tool for real-world content production. The code is available at: https://github.com/Eyeline-Labs/ID-V2V.

Video diffusion Model, Video-to-video generation, Video editing, Relighting

SIGGRAPH Asia 2026 Conference Papers; December 1–4, 2026; Kuala Lumpur, Malaysia

SIGGRAPH Asia 2026 Conference Papers (SA Conference Papers ’26), December 1–4, 2026, Kuala Lumpur, Malaysia

1. Introduction

In visual content creation workflows, a highly effective strategy is to capture characters and their performance first, and then design the visual appearance of the video later during post-production. By decoupling performance capture from visual design, this workflow allows creators to preserve character performance—such as facial expressions, motion, and interactions—with high fidelity, while adding or modifying visual elements afterward, including background, lighting, and overall style. This separation also avoids the need to physically construct the final visual setup during capture, reducing constraints from location, set design, and on-set lighting. Together, these properties establish a powerful paradigm centered on faithful preservation of character performance while enabling flexible post-hoc design of the visual presentation. Meanwhile, recent advances in video generation models (Blattmann et al., 2023b, a; Guo et al., 2024; Yang et al., 2024b; Team, 2025; HaCohen et al., 2026), have shown remarkable progress in synthesizing high-quality, temporally coherent videos. This naturally raises the question of whether generative video models can enable a similar human-centric paradigm: faithfully preserving character performance—often crucial for creative intent and narrative meaning—while flexibly generating the visual context of the scene.

We formalize this challenge as identity-preserving video restylization, a task that propagates scene, lighting, and appearance changes specified by an edited keyframe across a source video, while preserving facial likeness and performance, including expressions, eye gaze, and lip synchronization. Human subjects may be relit to match the edited scene, while their identity and performance remain unchanged. The target appearance is specified via an edited keyframe, which provides an intuitive and precise control interface that aligns naturally with existing image-based editing tools used in professional workflows (Wu et al., 2025; Fortin et al., 2025). This formulation allows productions to capture performances with minimal physical setup and subsequently define the desired visual appearance through keyframe editing, enabling the generation of a consistently restylized video while preserving the original human performance.

However, a central challenge of this task is the lack of large-scale paired training data in real-world settings, as it is difficult to obtain two videos showing the same character performing the identical actions under different visual styles. To overcome this challenge, our key insight is a principled decoupling between edit-driven synthesis and source-grounded identity preservation. Edit-driven synthesis should remain flexible and creative with respect to the edited keyframe, allowing the model to freely generate the visual content specified by the edit. In contrast, identity preservation must be tightly constrained by the source video. Crucially, we observe that under identity-preserving restylization, facial structure and expression should remain invariant, with illumination constituting the primary source of permissible variation. This observation leads us to formulate identity preservation as a video relighting problem (Debevec et al., 2000), while instantiating edit-driven synthesis as conditioned image-to-video generation.

Building on this observation, we propose ID-V2V, a video-to-video generative framework that addresses the data scarcity challenge by constructing dedicated control signals from a single training video with distinctive visual style, enabling both edit-driven synthesis and source-grounded identity preservation. Specifically, from the training video we derive relit face regions to simulate changes in facial lighting that may occur between the input video and the stylized output at inference time. We also extract face normal maps as complementary geometric cues. To represent the desired visual edits, we use the first frame of the training video together with a depth sequence extracted from it, which specify the target visual appearance while preserving the underlying scene geometry and motion. Together, these signals allow us to synthesize paired supervision from a single training video, effectively avoiding the need for scarce real-world video pairs for identity-preserving video restylization.

Our experiments on video restylization demonstrate two key advantages of ID-V2V. First, ID-V2V significantly outperforms existing baselines in preserving facial identity and fine-grained facial performance, including expressions and lip synchronization. In contrast, prior methods (Hu, 2024; Cheng et al., 2025; Wang et al., 2025; Jiang et al., 2025) typically rely on abstract control signals such as facial landmarks or low-dimensional identity embeddings, which are insufficient to faithfully retain detailed performance cues. Both our quantitative metrics and user study consistently support this finding. Second, ID-V2V reliably supports multi-subject video restylization while maintaining identity preservation for each individual and facial interaction, a capability that remains challenging for existing approaches yet is critical for real-world content creation scenarios.

In summary, our key contributions are as follows:

1. (1)

   We introduce identity-preserving video restylization, a new task motivated by real-world content creation workflows, where visual edits such as background and lighting are specified via an edited keyframe and coherently propagated across a source video while strictly preserving human identity and fine-grained performance.

2. (2)

   We propose ID-V2V, a video-to-video generative framework that addresses this task through a principled decoupling between edit-driven synthesis and source-grounded identity preservation. By formulating identity preservation as a video relighting problem and designing complementary control signals, it enables flexible edit propagation while tightly constraining human appearance and performance.

3. (3)

   Extensive experiments demonstrate that ID-V2V consistently outperforms existing baselines in preserving facial identity and fine-grained performance, robustly supports multi-subject scenarios, and achieves strong overall visual quality.

2. Related work

Identity-Preserving Generation with Reference Images.

A substantial body of prior work addresses identity preservation in generative video synthesis by conditioning on reference images or learned identity features, rather than an input performance video. Methods such as ConsisID (Huang et al., 2024a), EchoVideo (Wei et al., 2025), MagicMirror (Zhang et al., 2025b), ID-Animator (He et al., 2024b), Phantom (Liu et al., 2025a), HuMo (Chen et al., 2025), and VirtuallyBeing (Xu et al., 2025) , IRPO (Shen et al., 2026), Stand-In (Xue et al., 2026), and IP-FaceDiff (Anand et al., 2025), primarily operate in text-to-video or image-to-video settings, where one or a few reference images are used to inject identity cues into the generation process. While these approaches improve visual identity consistency, they do not take a source video as input and thus cannot exploit temporal cues from real performances, such as subtle facial expressions and performances. As a result, identity preservation is often limited to coarse appearance, and alignment with a captured performance remains underconstrained. They also typically lack artist-driven editing interfaces, such as edited keyframes, limiting fine-grained and controllable restylization in practical production settings.

Video-to-Video Generation and Performance Transfer.

Several video-to-video generation methods (Burgert et al., 2025; Jiang et al., 2025) aim to transfer motion or structural information from an input video through intermediate control signals. Approaches such as Wan-Animate (Cheng et al., 2025), AnimateAnyone (Hu, 2024), VACE (Jiang et al., 2025), FantasyPortrait (Wang et al., 2025), and SteadyDancer (Zhang et al., 2025a) infer facial identity primarily from a single input frame and use facial or motion control signals to guide video generation. Because identity cues are not grounded in the full source video, these methods struggle with multi-view identity preservation when the viewing direction changes, or must implicitly infer identity under variations in expression and lighting, which can lead to inaccurate identity reconstruction. In addition, facial performance is typically driven by abstract or low-dimensional representations—such as facial pose or landmarks (e.g., VACE, AnimateAnyone, SteadyDancer) or compact facial and expression embeddings (e.g., FantasyPortrait, Wan-Animate)—rather than directly leveraging the source video. Consequently, while these approaches can generate temporally coherent facial motion, they often fail to faithfully preserve fine-grained facial performance details, including subtle expressions, eye gaze, and lip synchronization, which are critical for identity-preserving video restylization. More broadly, these methods are not explicitly designed for this setting, as they prioritize motion or structural transfer over strict identity and performance fidelity.

General Stylization.

General image and video stylization transforms the visual appearance of the input according to an artistic reference or edited exemplar, enabling effects such as cartoonization, oil-painting styles, or other user-specified appearances (Gatys et al., 2016; Johnson et al., 2016; Huang and Belongie, 2017; Ruder et al., 2016; Jamriška et al., 2019; Spetlik et al., 2025). Such methods may stylize the face or the entire human subject, potentially altering identity-defining appearance and fine-grained performance details. In contrast, identity-preserving video restylization imposes a stricter constraint: the subject may only be relit, while their intrinsic identity and performance must be faithfully preserved.

Relighting and Illumination Control.

Relighting is an active area of research that aims to manipulate illumination conditions while preserving the underlying visual content. A substantial body of work studies single-image portrait relighting and facial illumination manipulation, focusing on modifying facial lighting while preserving human identity (He et al., 2024a; Mei et al., 2025; Rao et al., 2024; Ponglertnapakorn et al., 2023; Mei et al., 2024). Related methods also address portrait relighting for background replacement, lighting harmonization, and shadow removal (Pandey et al., 2021; Ren et al., 2024; Yoon et al., 2024; Kim et al., 2024). These approaches leverage human face priors, real captured data, or physically based rendering models, but are primarily designed for single images and can exhibit temporal inconsistencies when applied to video. Recently, more works have been proposed to address relighting and illumination editing in videos, with an emphasis on temporal consistency across frames. Representative methods include Light-X (Liu et al., 2025b), Lumen (Zeng et al., 2025), Light-A-Video (Zhou et al., 2025), RelightVide (Fang et al., 2025), RelightMaster (Bian et al., 2025) and UniRelight (He et al., 2025). These methods modify video lighting while maintaining temporal coherence. However, they mainly rely on text-based or parametric controls and do not support edited keyframes as explicit user input. They also do not target identity-preserving restylization, where identity and performance must be preserved under substantial visual changes beyond lighting. We therefore leverage portrait image relighting as a control signal that changes lighting while preserving facial features and identity.

3. Method

3.1. Task Definition

Task Inputs and Output.

We study keyframe-based identity preserving video restylization, illustrated in 1. The task takes two primary inputs. (1) A source video that contains a human performance. This video may be captured in a simple or different environment from the desired final output, but the goal is to faithfully preserve the character’s identity, including both facial likeness and dynamic facial performance throughout the sequence. (2) An edited first frame, obtained by modifying the first frame of the source video to specify the desired visual edits, such as changes to the environment, lighting, or other visual elements. These edits can be created manually by artists or generated using existing image editing models, providing an intuitive interface for visual control. Given these inputs, the task is to propagate the visual edits specified by the keyframe across the entire video while strictly preserving the character’s facial likeness and performance in the generated video.

Human-centric Content Creation.

By enabling intuitive first-frame edits that automatically propagate across the entire video, this formulation provides a convenient interface for creators to redesign scenes, environments, lighting, and other visual elements after the original performance has been captured. At the same time, it treats the human subject and their performance as invariant, faithfully preserving facial identity and expressive dynamics throughout the video. This aligns with a fundamental requirement in many forms of content creation, where human performance anchors storytelling and artistic intent.

Challenge in Identity Preservation.

Preserving facial likeness and fine grained facial performance under video restylization is inherently difficult, because these attributes are encoded in subtle cues such as lip synchronization, micro expressions, eye gaze, and emotional dynamics, which can be easily disrupted by visual changes. This challenge arises from both modeling and data limitations. (1) Modeling challenge. Such subtle performance cues are poorly captured by abstract representations such as depth maps or identity embeddings, causing existing methods to degrade identity and expression fidelity. (2) Data challenge. Large scale paired training data for this task is scarce, as it is difficult to obtain two videos showing the same character performing identical actions under different visual styles.

3.2. Decoupling Edit-Driven Synthesis and Identity Preservation

Overview.

Our key insight is to decouple edit-driven synthesis from source-grounded identity preservation. The synthesis component determines what visual content to generate and is guided by the edited keyframe, allowing flexible generation of the desired scene appearance. In contrast, identity preservation specifies what must remain unchanged: the human subject is tightly anchored to the source video to faithfully retain facial likeness and fine grained facial performance. Guided by this separation, ID V2V employs two complementary types of control signals, illustrated in Figure 2. Relit face video and face normal maps enforce identity preservation, while the first frame and depth sequence guide video synthesis. This design enables the construction of effective training pairs, thereby overcoming the scarcity of real-world before-and-after restylization examples.

Edit-Driven Synthesis.

This component translates keyframe edits into a temporally coherent video. We implement it as image to video generation with depth control. The model takes the edited keyframe together with a depth sequence extracted from the source video using DepthAnything 2 (Yang et al., 2024a). The edited keyframe provides explicit guidance on the desired appearance and stylistic changes, while the depth signal encodes global motion and scene layout. Together, these signals enable consistent propagation of complex visual edits across time while preserving motion coherence and scene geometry.

Identity Preservation as Relighting.

Preserving human identity and fine grained facial performance is particularly challenging in video restylization. These attributes are encoded in subtle, high frequency visual cues that are poorly captured by abstract control signals such as depth maps or facial landmarks. Instead, faithful preservation requires directly leveraging the pixel level information of human regions in the source video. A key observation is that identity preserving restylization should keep facial structure and expression unchanged, while illumination accounts for most permissible variation. Therefore, we formulate identity preservation as a video relighting problem, where the lighting may change while the underlying facial identity and performance remain fixed.

Relit Face Control.

Following the relighting formulation for identity preservation, we condition the model on relit face and head regions to explicitly model lighting changes on the human subject. Generating such relit data requires an accurate relighting model that can modify facial illumination while preserving fine details of the face and head with high fidelity. To meet this requirement, we train a dedicated image based face relighting model. We then apply this model frame by frame to the detected face and head regions of the training video, producing relit face sequences that serve as conditioning signals during training. This design effectively adapts image based relighting as a control signal for video restylization, providing a principled mechanism for preserving identity.

Relighting Model Design.

The design of our relighting model is inspired by the image de-lighting approach of LuxPostFacto (Mei et al., 2025). However, unlike the flat lit normalization used in LuxPostFacto, we relight faces from the training video to a regular lighting condition. This choice is motivated by the training setup of ID-V2V: the relit faces serve as inputs during training and therefore should resemble the lighting conditions typically encountered at inference time, where source videos are often captured under ordinary lighting. Meanwhile, faces in the training videos frequently exhibit cinematic or stylized illumination, which we treat as the desired target appearance. As a result, ID-V2V learns to transform regular facial illumination into cinematic or stylized lighting for the human regions. In practice, we simulate regular illumination using a top lighting configuration. As shown in fig. 6, the relighting model removes the original illumination and applies consistent regular lighting to the face regions, producing reliable conditioning signals for facial likeness and performance.

Relighting Model Training.

To train the relighting model, we construct a large scale dataset of paired face images using the One Light at a Time (OLAT) (Debevec et al., 2000) data collected in LuxPostFacto (Debevec et al., 2000). For each pair, the input image is a face rendered under arbitrary environmental illumination, synthesized by compositing OLAT captures with HDR environment maps randomly sampled from PolyHeaven (Poly Haven, 2023), following the image relighting procedure of (Debevec et al., 2000). The target image is the same face relit with a vertically half white, half black HDRI that produces the desired top lighting configuration. Using data from 67 participants with diverse head poses, facial expressions, and camera viewpoints, this process yields approximately 330,000 image pairs. The model architecture and training settings largely follow the image relighting model of LuxPostFacto (Mei et al., 2025). Specifically, we adopt the pretrained SVD model (Blattmann et al., 2023a) as the base network and reduce the number of frames to one to convert it into an image based model. We further modify the model to condition on an HDR map and fine tune it on the paired training images for 200K steps. During inference, we detect and crop face regions from each video frame and apply the relighting model independently to produce their top lit versions.

Face Normals Control.

While relit faces provide rich cues about facial appearance under varying illumination, they remain insufficient to fully constrain facial geometry due to the inherent ambiguity between lighting and surface shape, which can lead to geometric drift and identity degradation. To explicitly encode facial structure and stabilize fine-grained identity cues, we additionally condition on surface normal maps predicted using the DAViD (Saleh et al., 2025) model. Relighting enforces appearance consistency, while normals provide complementary geometric constraints on facial shape. Together, they form a complete and robust control signal for faithful preservation of facial identity and performance under stylistic changes.

3.3. Architecture, Training and Inference

Model Architecture and Training.

To support multiple spatially aligned control videos, we build upon VACE (Jiang et al., 2025) based on Wan2.1 (Team, 2025), which natively supports video conditioning, and extend it to a multi ControlNet (Zhang et al., 2023) architecture (Figure 3). To reduce computational overhead, each control signal is processed using the same VACE context block, and the resulting features are fused by summation before being injected into the main DiT blocks. For keyframe control, we follow the VACE input format and replace the first frame in the control videos with the edited first frame. The model also conditions on a text prompt and is trained jointly with all control signals.

Control Signals During Inference.

During inference, we directly use the original facial region pixels from the source video to avoid any loss of identity information. This follows the training design, where relit faces are used only during training to simulate the lighting conditions of the source video at inference. As a result, the model learns to transform the regular facial lighting of the input source video into the stylized illumination specified by the edited first frame. Although the model is trained with pixel and normal guidance restricted to the facial region, the guided region can be expanded at inference time. Providing full-body pixels and normals helps preserve the appearance and structure of the entire body, while extending the guidance to the full frame can further enable full-scene relighting to some extent. The global depth video is kept the same as during training to maintain geometric consistency and coherent propagation of the visual edits across the sequence.

4. Experiments

4.1. Experimental Setup

Training.

We collect a dataset of 40k human-centric videos featuring both single- and multi-subject scenarios, covering diverse scenes, lighting, actions, and facial expressions. To support the post-training requirements of VACE, we generate textual descriptions for each video using the QwenVL2.5-32B (Bai et al., 2025) model. For identity preservation, we run SCRFD face detection (Guo et al., 2021) on all videos and retain up to five faces per frame, selected by pixel area, to accommodate multi-subject content commonly encountered in real-world content creation. As a result, the relit-face and normal-based control signals contain at most five faces per frame, even when additional faces are present. All models are trained at a resolution of 1280 × 720 with 81 frames per video. To ensure effective learning of identity-related cues, we discard videos in which more than 40% of frames contain no detected faces.

Evaluation Datasets.

We construct two human-centric evaluation datasets: 100 single-subject videos and 60 two-subject videos, covering diverse actions and facial expressions. We report results separately on these two settings. For quantitative evaluation, we restrict the maximum number of subjects to two, as existing baselines are unable to reliably handle more subjects; nevertheless, ID-V2V naturally supports multi-subject scenarios, which we additionally demonstrate qualitatively.

To specify the target style, we first use Gemini-2.5 (Comanici et al., 2025) to analyze the original first frame and propose a target background and lighting configuration. These proposed edits are then executed by an image editing model, Qwen-Image-Edit (Wu et al., 2025), with depth and Canny edge conditions, producing substantial changes in background and illumination. We further perform manual filtering to ensure high-quality edited keyframes. Each evaluation sample therefore consists of a source video and an edited first frame, and we assess how well different methods propagate the specified edits across the full video while preserving human facial likeness and performance.

Baselines.

We select baseline methods that take an edited first frame as input and provide facial or motion control to guide video generation. Based on this criterion, we compare against AnimateAnyone (Hu, 2024), a UNet-based method, as well as several DiT-based approaches built on Wan model’s backbones, including VACE (Jiang et al., 2025), FantasyPortrait (Wang et al., 2025), and SteadyDancer (Zhang et al., 2025a), and WanAnimate (Cheng et al., 2025).

4.2. Evaluation Metrics

Facial Likeness.

To evaluate facial likeness between the generated video and the source video, we apply SCRFD face detection (Guo et al., 2021) to extract face crops and compute identity similarity using AdaFace (Kim et al., 2022). Frames without detected faces are discarded. For videos with multiple subjects, we resolve face correspondence across videos using AdaFace similarity scores to consistently match each generated face with its corresponding reference identity.

Facial Performance Preservation.

We measure the similarity of facial expressions using two widely adopted expression representations. The first is Action Units (AUs) from the Facial Action Coding System (FACS) (Ekman and Friesen, 1978), which capture fine-grained facial muscle activations. The second is expression vectors extracted from EMOCA (Daněček et al., 2022), an emotion-aware 3D face reconstruction model that estimates detailed facial expression parameters. For each face in the generated video, we compute the cosine similarity between its expression representation (either AU vectors or EMOCA expression vectors) and that of the corresponding face in the source video, denoted as Exp-AU and Exp-EMOCA, respectively. These metrics provide complementary evaluations of facial expression preservation.

4.3. Results

媒体内容 · 前往原文查看

Method	Facial Metrics					VBench & VBench2.0				UserStudy
	AdaFace	Exp- AU	Exp- EMOCA	CLIP-T	CLIP-I	Subject Consistency	Background Consistency	Temporal Flickering	Human Anatomy	Facial Likeness	Facial Perf. Preservation	Quality
Single-subject video restylization
AnimateAnyone	0.402	0.477	0.690	0.331	0.976	0.961	0.951	0.971	0.923	0.23%	0.23%	0.23%
FantasyPortrait	0.507	0.550	0.767	0.332	0.997	0.953	0.950	0.991	0.930	1.82%	2.27%	2.28%
VACE	0.465	0.580	0.776	0.334	0.992	0.939	0.932	0.985	0.923	7.95%	8.18%	10.05%
WanAnimate	0.529	0.600	0.841	0.344	0.993	0.962	0.945	0.987	0.941	10.45%	11.36%	12.56%
SteadyDancer	0.479	0.554	0.745	0.328	0.993	0.955	0.945	0.985	0.931	4.55%	3.64%	4.79%
Ours	0.701	0.668	0.899	0.344	0.993	0.960	0.954	0.993	0.945	75.00%	74.32%	70.09%
Ours (w/o face normals)	0.676	0.638	0.881	0.343	0.992	0.958	0.948	0.990	0.943	-	-	-
Ours (w/o face video)	0.574	0.585	0.798	0.344	0.993	0.955	0.952	0.992	0.938	-	-	-
Two-subject video restylization
FantasyPortrait	0.433	0.624	0.747	0.331	0.997	0.930	0.929	0.986	0.885	0.65%	0.22%	1.31%
VACE	0.401	0.639	0.796	0.329	0.994	0.956	0.947	0.987	0.876	9.80%	10.00%	12.42%
WanAnimate	0.432	0.617	0.732	0.330	0.995	0.975	0.954	0.988	0.921	3.92%	3.70%	5.23%
Ours	0.637	0.709	0.901	0.331	0.996	0.979	0.956	0.988	0.926	85.62%	86.09%	81.05%

General Video Quality.

We evaluate general video quality along three complementary aspects: prompt alignment, temporal consistency, and overall visual fidelity. Prompt alignment is assessed using CLIP-T (Radford et al., 2021) by computing the average CLIP similarity between the input prompt and generated frames. Temporal consistency is measured via the average CLIP image similarity between consecutive frames. To further assess video fidelity and perceptual quality, we benchmark our method using VBench (Huang et al., 2024b), including aesthetic quality, imaging quality, subject consistency, background consistency, temporal style, and VBench-2.0 (Zheng et al., 2025) to evaluate human anatomy correctness.

User Study.

We conduct a user study on both the single-subject and two-subject evaluation datasets, each containing 20 samples, with 26 participants to compare video restylization methods. For each example, participants are shown the source video and the edited first frame, followed by videos generated by different methods, and are asked to select the best result under each criterion. The study includes three evaluation criteria: (i) Identity Similarity, which method makes the generated person look most like the source person; (ii) Expression Preservation, which method best preserves facial performance, including lip synchronization, eye-gaze direction, and subtle expressions and emotions; and (iii) Visual Quality, which method produces the best overall visual quality.

Qualitative Results.

Visual comparisons between ID-V2V and baseline methods are shown in fig. 4 for the single-subject case and in fig. 7 for the two-subject case, with frame-by-frame comparisons. In the single-subject setting, ID-V2V preserves facial likeness more faithfully than the baselines, retaining fine-grained details such as teeth shape, subtle facial muscle movements (e.g., eyebrow motion during speech), and eye blinks, at a level of fidelity not achieved by competing methods. Notably, ID-V2V also handles face occlusion effectively, as illustrated in the left example of fig. 4, maintaining consistent facial performance before and after the occlusion. In contrast, FantasyPortrait produces corrupted results in occlusion scenarios. In the two-subject setting, methods that rely on compact facial or expression representations (e.g., WanAnimate and FantasyPortrait) struggle to preserve both facial identity and expression, with degradation becoming more pronounced when multiple subjects are present. In contrast, ID-V2V better preserves individual identities and facial performances, which in turn allows it to maintain the interactions between subjects in the source video.

Quantitative Results on Identity Preservation.

Quantitative results are reported in table 1. We evaluate facial identity preservation using the AdaFace score and assess facial performance preservation using Exp-AU and Exp-EMOCA. Across both the single-subject and two-subject settings, ID-V2V consistently outperforms all baseline methods, highlighting its ability to preserve both facial identity and fine-grained facial performance under video restylization.

Quantitative Results on General Video Quality.

As shown in table 1, ID-V2V achieves superior or, in most cases, comparable performance to baseline methods on general video quality metrics. This indicates that training the model for identity-preserving video restylization does not compromise its overall video generation capability. Importantly, baseline methods that are primarily designed for face-specific generation (e.g., FantasyPortrait and WanAnimate) exhibit noticeable degradation in general video quality when moving from single-subject to two-subject scenarios. In contrast, ID-V2V maintains strong performance in both settings, underscoring its robustness and effectiveness for multi-subject video generation.

User Study.

User study results are reported in table 1. Across all evaluation criteria—including facial appearance preservation, facial performance preservation, and overall visual quality—ID-V2V is consistently preferred over all baseline methods. In the single-subject setting, ID-V2V achieves win rates exceeding 74% on both facial likeness and facial performance, and over 70% on overall video quality. The advantage becomes even more pronounced in the two-subject setting, where ID-V2V attains over 85% win rates on facial likeness and performance, and more than 80% on overall video quality. These results show that ID-V2V more effectively preserves identity and facial performance with high-quality video generation, particularly in challenging two-subject scenarios.

More Subjects.

Results demonstrating ID-V2V on videos with more than two subjects are shown in fig. 8. ID-V2V consistently preserves each subject’s facial appearance and expressions, as well as the interactions between subjects, while restylizing the video into a new scene with different visual elements. These results indicate that ID-V2V effectively supports multi-subject video restylization, where maintaining inter-subject facial performance and interaction is often critical for conveying narrative and semantic content in real-world applications.

Ablation Study on Facial Conditions.

We ablate the two facial conditions in ID-V2V: relit face videos and face normals. Specifically, we train two variants, one without face normals and one without relit face videos. Quantitative results are reported in table 1. Removing the relit face video condition causes a significant drop in facial preservation metrics, showing that directly leveraging facial pixels is crucial for maintaining fine-grained facial details. Removing face normals also degrades facial metrics, although to a lesser extent, suggesting that geometric facial cues provide complementary guidance. Together, these results show that training with relit face videos and face normals jointly achieves strong identity and fine-grained facial performance preservation.

Failure mode: extreme source lighting.

A failure mode occurs under extreme or irregular source lighting, such as strong colored illumination or hard shadows. As shown in fig. 5, although the edited first frame removes the green illumination, residual lighting cues reappear in later generated frames, resulting in an unintended color cast and temporally inconsistent appearance. For more reliable results, we recommend using source videos captured under relatively regular lighting, which better matches the conditions represented in the training data.

Relighting.

Although ID-V2V is trained using facial relighting supervision, it can be naturally extended to full-scene relighting at inference time. During inference, instead of providing only the facial region as the relighting guidance, we directly use the entire source video as input, allowing the model to exploit appearance cues from the full scene. As shown in fig. 9, ID-V2V not only relights the face but also consistently relights the body, surrounding objects, and the background. Despite never being explicitly trained for full-scene relighting, the learned facial relighting capability generalizes well beyond the face, producing coherent illumination changes across the entire scene.

5. Conclusion and Limitations

In this paper, we address a fundamental challenge in generative video for content production: preserving faithful human identity and performance, both of which are central to creative intent and narrative meaning, while still enabling flexible visual edits to the scene. We formalize this challenge as identity-preserving video restylization, a human-centric task that propagates visual edits specified by an edited keyframe across a source video while strictly maintaining facial likeness and fine-grained facial performance such as expressions, gaze, and subtle motion dynamics. Unlike conventional video editing or style transfer, the goal is not merely visual consistency but the faithful preservation of a person’s identity and acting performance from a source video. Crucially, real production settings frequently involve multiple interacting subjects, where maintaining each individual’s identity as well as the temporal consistency of their facial interactions is essential for preserving semantic meaning and narrative coherence. To overcome the lack of paired training data inherent to this task, we propose ID-V2V, a video-to-video framework that decouples source-grounded identity preservation from edit-driven video synthesis, enabling the effective construction of training pairs from a single video without requiring explicit paired supervision. This decoupling is realized through complementary control signals: relit facial regions and facial normal maps tightly constrain identity and performance, while edited keyframes and depth sequences guide flexible, spatially consistent, and temporally coherent generation. Extensive experiments demonstrate that ID-V2V substantially outperforms existing methods in preserving facial identity, subtle performance cues, and inter-subject interactions, while robustly supporting multi-subject scenarios and maintaining high overall video quality. Together, these results highlight the promise of ID-V2V as a practical and scalable human-centric approach for real-world content production.

Limitations.

Our method has two main limitations. First, because the source depth sequence provides guidance for scene structure and motion, large geometric discrepancies between the source video and the edited keyframe can make this conditioning overly restrictive. Moderate background changes are generally tolerated, but extreme differences in scene layout or depth may conflict with the source geometry and reduce edit fidelity. This limitation could be mitigated by training with depth dropout, allowing the model to optionally operate without depth guidance at inference time when the target scene differs substantially from the source. Second, the method is designed for source videos captured under relatively regular lighting, consistent with both its training conditions and the intended capture-first, restyle-later workflow. Under extreme or irregular illumination, such as strong colored lighting or hard shadows, residual source-lighting cues may propagate into the generated video, producing unintended color casts or temporally inconsistent appearance.

Acknowledgements.

We would like to thank Lukas Lepicovsky, Mohsen Mousavi, David Rhodes, Nhat Phong Tran, Dacklin Young, and Johnson Thomasson for their production insights. We also thank Jeffrey Shapiro and Ritwik Kumar for their executive support, and Jennifer Lao and Lianette Alnaber for their operational support.

References

• T. Anand, A. Garg, and K. Mitra (2025) IP-FaceDiff: identity-preserving facial video editing with diffusion. In Proceedings of the Winter Conference on Applications of Computer Vision Workshops, pp. 248–258. Cited by: §2.
• S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. (2025) Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: §4.1.
• W. Bian, X. Shi, Z. Huang, J. Bai, Q. Wang, X. Wang, P. Wan, K. Gai, and H. Li (2025) Relightmaster: precise video relighting with multi-plane light images. arXiv preprint arXiv:2511.06271. Cited by: §2.
• A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts, et al. (2023a) Stable video diffusion: scaling latent video diffusion models to large datasets. arXiv. Cited by: §1, §3.2.
• A. Blattmann, R. Rombach, H. Ling, T. Dockhorn, S. W. Kim, S. Fidler, and K. Kreis (2023b) Align your latents: high-resolution video synthesis with latent diffusion models. In CVPR, Cited by: §1.
• R. Burgert, Y. Xu, W. Xian, O. Pilarski, P. Clausen, M. He, L. Ma, Y. Deng, L. Li, M. Mousavi, et al. (2025) Go-with-the-flow: motion-controllable video diffusion models using real-time warped noise. CVPR. Cited by: §2.
• L. Chen, T. Ma, J. Liu, B. Li, Z. Chen, L. Liu, X. He, G. Li, Q. He, and Z. Wu (2025) Humo: human-centric video generation via collaborative multi-modal conditioning. arXiv preprint arXiv:2509.08519. Cited by: §2.
• G. Cheng, X. Gao, L. Hu, S. Hu, M. Huang, C. Ji, J. Li, D. Meng, J. Qi, P. Qiao, et al. (2025) Wan-animate: unified character animation and replacement with holistic replication. arXiv preprint arXiv:2509.14055. Cited by: §1, §2, §4.1.
• G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. (2025) Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. Cited by: §4.1.
• R. Daněček, M. J. Black, and T. Bolkart (2022) Emoca: emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20311–20322. Cited by: §4.2.
• P. Debevec, T. Hawkins, C. Tchou, H. Duiker, W. Sarokin, and M. Sagar (2000) Acquiring the reflectance field of a human face. In SIGGRAPH, Cited by: §1, §3.2.
• P. Ekman and W. V. Friesen (1978) Facial action coding system. Environmental Psychology & Nonverbal Behavior. Cited by: §4.2.
• Y. Fang, Z. Sun, S. Zhang, T. Wu, Y. Xu, P. Zhang, J. Wang, G. Wetzstein, and D. Lin (2025) RelightVid: temporal-consistent diffusion model for video relighting. arXiv preprint arXiv:2501.16330. Cited by: §2.
• A. Fortin, G. Vernade, K. Kampf, and A. Reshi (2025) Introducing gemini 2.5 flash image, our state-of-the-art image model. URL https://developers.googleblog.com/en/introducing-gemini-2-5-flash-image/. Cited by: §1.
• L. A. Gatys, A. S. Ecker, and M. Bethge (2016) Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423. Cited by: §2.
• J. Guo, J. Deng, A. Lattas, and S. Zafeiriou (2021) Sample and computation redistribution for efficient face detection. arXiv. Cited by: §4.1, §4.2.
• Y. Guo, C. Yang, A. Rao, Z. Liang, Y. Wang, Y. Qiao, M. Agrawala, D. Lin, and B. Dai (2024) Animatediff: animate your personalized text-to-image diffusion models without specific tuning. In ICLR, Cited by: §1.
• Y. HaCohen, B. Brazowski, N. Chiprut, Y. Bitterman, A. Kvochko, A. Berkowitz, D. Shalem, D. Lifschitz, D. Moshe, E. Porat, et al. (2026) LTX-2: efficient joint audio-visual foundation model. arXiv preprint arXiv:2601.03233. Cited by: §1.
• K. He, R. Liang, J. Munkberg, J. Hasselgren, N. Vijaykumar, A. Keller, S. Fidler, I. Gilitschenski, Z. Gojcic, and Z. Wang (2025) UniRelight: learning joint decomposition and synthesis for video relighting. arXiv preprint arXiv:2506.15673. Cited by: §2.
• M. He, P. Clausen, A. L. Taşel, L. Ma, O. Pilarski, W. Xian, L. Rikker, X. Yu, R. Burgert, N. Yu, et al. (2024a) DifFRelight: diffusion-based facial performance relighting. In SIGGRAPH Asia, Cited by: §2.
• X. He, Q. Liu, S. Qian, X. Wang, T. Hu, K. Cao, K. Yan, and J. Zhang (2024b) Id-animator: zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275. Cited by: §2.
• L. Hu (2024) Animate anyone: consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8153–8163. Cited by: §1, §2, §4.1.
• J. Huang, X. Dong, W. Song, Z. Chong, Z. Tang, J. Zhou, Y. Cheng, L. Chen, H. Li, Y. Yan, et al. (2024a) Consistentid: portrait generation with multimodal fine-grained identity preserving. arXiv. Cited by: §2.
• X. Huang and S. Belongie (2017) Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510. Cited by: §2.
• Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, et al. (2024b) Vbench: comprehensive benchmark suite for video generative models. In CVPR, Cited by: §4.3.
• O. Jamriška, Š. Sochorová, O. Texler, M. Lukáč, J. Fišer, J. Lu, E. Shechtman, and D. Sýkora (2019) Stylizing video by example. ACM Transactions on Graphics 38 (4), pp. 107:1–107:11. Cited by: §2.
• Z. Jiang, Z. Han, C. Mao, J. Zhang, Y. Pan, and Y. Liu (2025) Vace: all-in-one video creation and editing. arXiv preprint arXiv:2503.07598. Cited by: §1, §2, Figure 3, Figure 3, §3.3, §4.1.
• J. Johnson, A. Alahi, and L. Fei-Fei (2016) Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, pp. 694–711. Cited by: §2.
• H. Kim, M. Jang, W. Yoon, J. Lee, D. Na, and S. Woo (2024) SwitchLight: co-design of physics-driven architecture and pre-training framework for human portrait relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 25096–25106. Cited by: §2.
• M. Kim, A. K. Jain, and X. Liu (2022) Adaface: quality adaptive margin for face recognition. In CVPR, Cited by: §4.2.
• L. Liu, T. Ma, B. Li, Z. Chen, J. Liu, Q. He, and X. Wu (2025a) Phantom: subject-consistent video generation via cross-modal alignment. arXiv preprint arXiv:2502.11079. Cited by: §2.
• T. Liu, Z. Chen, Z. Huang, S. Xu, S. Zhang, C. Ye, B. Li, Z. Cao, W. Li, H. Zhao, et al. (2025b) Light-x: generative 4d video rendering with camera and illumination control. arXiv preprint arXiv:2512.05115. Cited by: §2.
• Y. Mei, M. He, L. Ma, J. Philip, W. Xian, D. M. George, X. Yu, G. Dedic, A. L. Taşel, N. Yu, V. M. Patel, and P. Debevec (2025) Lux post facto: learning portrait performance relighting with conditional video diffusion and a hybrid dataset. CVPR. Cited by: §2, §3.2, §3.2.
• Y. Mei, Y. Zeng, H. Zhang, Z. Shu, X. Zhang, S. Bi, J. Zhang, H. Jung, and V. M. Patel (2024) Holo-relighting: controllable volumetric portrait relighting from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2.
• R. Pandey, S. Orts-Escolano, C. Legendre, C. Haene, S. Bouaziz, C. Rhemann, P. E. Debevec, and S. R. Fanello (2021) Total relighting: learning to relight portraits for background replacement.. TOG. Cited by: §2.
• Poly Haven (2023) Poly haven. Note: https://polyhaven.comHDRI environments and 3D assets, CC0 license Cited by: §3.2.
• P. Ponglertnapakorn, Z. Zhang, and Q. Chen (2023) DiFaReli++: diffusion face relighting with consistent cast shadows. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2.
• A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In ICML, Cited by: §4.3.
• P. Rao, G. Fox, A. Meka, M. B R, F. Zhan, T. Weyrich, B. Bickel, H. Pfister, W. Matusik, M. Elgharib, and C. Theobalt (2024) Lite2Relight: 3d-aware single image portrait relighting. ACM Transactions on Graphics (TOG) 43 (4). Cited by: §2.
• M. Ren, W. Xiong, J. S. Yoon, Z. Shu, J. Zhang, H. Jung, G. Gerig, and H. Zhang (2024) Relightful harmonization: lighting-aware portrait background replacement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6452–6462. Cited by: §2.
• M. Ruder, A. Dosovitskiy, and T. Brox (2016) Artistic style transfer for videos. In German Conference on Pattern Recognition, pp. 26–36. Cited by: §2.
• F. Saleh, S. Aliakbarian, C. Hewitt, L. Petikam, X. Xiao, A. Criminisi, T. J. Cashman, and T. Baltrusaitis (2025) DAViD: data-efficient and accurate vision models from synthetic data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5348–5358. Cited by: §3.2.
• L. Shen, W. Jiang, Y. Zhu, J. Li, T. Ge, Z. Cao, and B. Zheng (2026) Identity-preserving image-to-video generation via reward-guided optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27281–27290. Cited by: §2.
• R. Spetlik, D. Futschik, and D. Sỳkora (2025) StructuReiser: a structure-preserving video stylization method. In Computer Graphics Forum, Vol. 44, pp. e70161. Cited by: §2.
• W. Team (2025) Wan: open and advanced large-scale video generative models. . Cited by: §1, §3.3.
• Q. Wang, M. Wang, F. Jiang, Y. Fan, Y. Qi, and M. Xu (2025) Fantasyportrait: enhancing multi-character portrait animation with expression-augmented diffusion transformers. arXiv preprint arXiv:2507.12956. Cited by: §1, §2, §4.1.
• J. Wei, S. Yan, W. Lin, B. Liu, R. Chen, and M. Guo (2025) EchoVideo: identity-preserving human video generation by multimodal feature fusion. arXiv preprint arXiv:2501.13452. Cited by: §2.
• C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, et al. (2025) Qwen-image technical report. arXiv preprint arXiv:2508.02324. Cited by: §1, §4.1.
• Y. Xu, W. Xian, L. Ma, J. Philip, A. Taşel, Y. Zhao, R. Burgert, M. He, O. Hermann, O. Pilarski, et al. (2025) Virtually being: customizing camera-controllable video diffusion models with volumetric performance captures. In Proceedings of the SIGGRAPH Asia 2025 Conference Papers, pp. 1–12. Cited by: §2.
• B. Xue, Z. Duan, Q. Yan, W. Wang, H. Liu, C. Guo, C. Li, C. Li, and J. Lyu (2026) Stand-in: a lightweight and plug-and-play identity control for video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2.
• L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao (2024a) Depth anything v2. Advances in Neural Information Processing Systems 37, pp. 21875–21911. Cited by: §3.2.
• Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, et al. (2024b) Cogvideox: text-to-video diffusion models with an expert transformer. arXiv. Cited by: §1.
• J. S. Yoon, Z. Shu, M. Ren, X. Zhang, Y. Hold-Geoffroy, K. K. Singh, and H. Zhang (2024) Generative portrait shadow removal. ACM Transactions on Graphics 43 (6), pp. 1–13. External Links: Document Cited by: §2.
• J. Zeng, Y. Liu, Y. Feng, C. Miao, Z. Gao, J. Qu, J. Zhang, B. Wang, and K. Yuan (2025) Lumen: consistent video relighting and harmonious background replacement with video generative models. arXiv preprint arXiv:2508.12945. Cited by: §2.
• J. Zhang, S. Cao, R. Li, X. Zhao, Y. Cui, X. Hou, G. Wu, H. Chen, Y. Xu, L. Wang, et al. (2025a) SteadyDancer: harmonized and coherent human image animation with first-frame preservation. arXiv preprint arXiv:2511.19320. Cited by: §2, §4.1.
• L. Zhang, A. Rao, and M. Agrawala (2023) Adding conditional control to text-to-image diffusion models. In ICCV, Cited by: §3.3.
• Y. Zhang, Y. Liu, B. Xia, B. Peng, Z. Yan, E. Lo, and J. Jia (2025b) Magicmirror: id-preserved video generation in video diffusion transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14464–14474. Cited by: §2.
• D. Zheng, Z. Huang, H. Liu, K. Zou, Y. He, F. Zhang, L. Gu, Y. Zhang, J. He, W. Zheng, et al. (2025) Vbench-2.0: advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755. Cited by: §4.3.
• Y. Zhou, J. Bu, P. Ling, P. Zhang, T. Wu, Q. Huang, J. Li, X. Dong, Y. Zang, Y. Cao, A. Rao, J. Wang, and L. Niu (2025) Light-a-video: training-free video relighting via progressive light fusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §2.