GenRecon:连接生成先验用于多视角三维场景重建
阅读原文· arxiv.org该方法提出一种高保真多视角三维场景重建方案,核心是将重建过程与强大的生成式3D先验紧密耦合。具体做法是将场景划分为多个空间局部重叠的区块进行条件化3D生成,并提出一种基于投影的条件机制,将多视角图像特征提升为与生成模型对齐的、空间锚定的连贯3D表示。该方法以Trellis.2等前沿生成模型为基础,将其对象级能力推广至场景级别,最终生成可编辑的PBR网格重建结果。在室内环境重建任务上,其保真度优于现有尖端方法16%。
We introduce a new approach to high-fidelity 3D scene reconstruction from multi-view RGB images that tightly couples reconstruction with a strong generative 3D prior. We cast scene reconstruction as conditional 3D generation over a set of spatially-localized, overlapping chunks that together tile the scene, scaling generation to large scene extents. Crucially, we inherit the fidelity and completeness of state-of-the-art generative shape models -- we use Trellis.2 as an example -- which we generalize to the scene level. To this end, we propose a projection-based conditioning mechanism that lifts posed multi-view image features into a coherent 3D representation aligned with the generative model, independent of view ordering and spatially anchored to the scene, yielding high-fidelity, multi-view consistent generated geometry. This enables lifting the strong object-level prior of Trellis.2 to multi-view, scene-scale generation, producing faithful, editable PBR mesh reconstructions of indoor environments. As a result, we obtain high-fidelity results that outperform cutting-edge reconstruction methods by 16%.