Free Geometry:从自身长序列中精炼3D重建
阅读原文· arxiv.orgFree Geometry框架通过自监督学习使前馈3D重建模型在测试时自我进化,无需3D真值标注。其核心洞察是:更多视图可产生更可靠的重建。通过掩码部分帧构建自监督任务,强制完整与部分观察的跨视图特征一致性,并采用LoRA实现快速重校准(单GPU不到2分钟)。在4个基准数据集上,该方法显著提升了Depth Anything 3和VGGT等模型的性能,相机位姿精度平均提升3.73%,点图预测精度提升2.88%。
Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually plausible reconstructions often contain errors, particularly under occlusions, specularities, and ambiguous cues. To address this, we introduce Free Geometry, a framework that enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth. Our key insight is that, when the model receives more views, it produces more reliable and view-consistent reconstructions. Leveraging this property, given a testing sequence, we mask a subset of frames to construct a self-supervised task. Free Geometry enforces cross-view feature consistency between representations from full and partial observations, while maintaining the pairwise relations implied by the held-out frames. This self-supervision allows for fast recalibration via lightweight LoRA updates, taking less than 2 minutes per dataset on a single GPU. Our approach consistently improves state-of-the-art foundation models, including Depth Anything 3 and VGGT, across 4 benchmark datasets, yielding an average improvement of 3.73% in camera pose accuracy and 2.88% in point map prediction. Code is available at https://github.com/hiteacherIamhumble/Free-Geometry .