Lift4D:调和单视图三维估计以实现野外四维重建
阅读原文· arxiv.orgLift4D 是一种测试时优化框架,用于从单目视频重建动态非刚性物体。它首先通过因果潜在条件适配单视图3D重建模型,生成时间一致的逐帧预测,作为可变形3D高斯溅射表示的初始化;随后通过遮挡感知优化与视图条件扩散先验,在恢复可见表面细节的同时补全未观测区域。在包含严重遮挡和非刚性运动的野外序列上,Lift4D 明显优于此前方法。
Reconstructing dynamic non-rigid objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior approaches either learn to directly predict 4D representations from visual input or initialize a 3D representation that is subsequently deformed and refined based on video evidence. However, the former are constrained by the scarcity of 4D training data, while the latter leverage priors only for the initial reconstruction and rely solely on video supervision thereafter; neither handles complex in-the-wild scenarios with large deformations and occlusions well. We present Lift4D, a test-time optimization framework that addresses both limitations. First, we adapt an existing single-view 3D reconstruction model to yield temporally consistent per-frame predictions via causal latent conditioning, providing a coherent initialization for a deformable 3D Gaussian Splatting representation. We then ``sculpt'' this representation to match the input video through an occlusion-aware optimization that faithfully recovers visible surface details while completing unobserved regions using a view-conditioned diffusion prior. We demonstrate that Lift4D clearly improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion.