TryOnCrafter:通过可渲染4D试穿代理释放相机轨迹实现真实视频虚拟试穿
阅读原文· arxiv.orgTryOnCrafter是首个面向相机可控视频虚拟试穿(CaM-VVT)的统一DiT框架。它引入可渲染4D试穿代理,将高保真2D试穿先验蒸馏为基于3DGS的服装化身,再由SMPL-X序列驱动动画并度量对齐至重建背景点云,实现人体与环境的显式解耦。基于该代理作为几何锚点,Proxy-Anchored Video DiT生成严格遵循指定相机轨迹和物理形变的光照真实视频。该框架还支持人体重定位、子弹时间效果和360度轨道观看等应用。
While Video Virtual Try-on (VVT) has achieved remarkable progress in synthesizing realistic garment overlays on dynamic subjects, existing paradigms remains fundamentally constrained by a passive dependency on source camera trajectories, failing to accommodate the requisite interactive freedom for omnidirectional viewpoint exploration. To address this limitation, we define a pioneering research frontier: Camera-controllable Video Virtual Try-on (CaM-VVT). Unlike conventional VVT, CaM-VVT not only necessitates viewpoint-agnostic texture hallucination but also strict structural synchronization between non-rigid human dynamics and background contexts under arbitrary, unconstrained camera movements. To tackle these challenges, we present TryOnCrafter, the first unified DiT-based framework specifically architected for the CaM-VVT task. Departing from implicit pixel-space manipulation, we introduce a Renderable 4D Try-on Proxy that explicitly decouples the human subject from the environment. This is achieved by distilling high-fidelity 2D try-on priors into a clothed 3DGS-based avatar, which is subsequently animated via SMPL-X sequences and metric-aligned into a reconstructed background point cloud. This proxy establishes a robust structural foundation with superior texture density and motion integrity. Our Proxy-Anchored Video DiT leverages this robust structural foundation as a primary geometric anchor, ensuring that the synthesized photorealistic videos are strictly constrained by prescribed trajectories and physically plausible deformations. Benefiting from the inherent editability of the 4D proxy, TryOnCrafter facilitates diverse downstream applications, including human relocalization, ``bullet time'' effects, and 360-degree orbital viewing.