UniSHARP: 通用单目视图合成方法
阅读原文· arxiv.orgUniSHARP扩展了SHARP真实感视图合成方法,实现从传统透视相机到鱼眼、全景等系统的通用单目渲染。核心思路是在统一全向潜空间中对齐图像,在射线基表示中沿射线和径向距离排列高斯原语,并联合解码UniK3D编码器提取的2D语义和3D空间特征。构建了覆盖多种成像系统并按视场角分层的benchmark,实验表明UniSHARP大幅优于替代方法。
In this work, we focus on extending SHARP, the popular photorealistic view synthesis method, for universal monocular rendering across a continuum of camera systems, from conventional perspective cameras to wide-field-of-view, fisheye and omnidirectional panoramic settings. To overcome the pinhole-specific assumptions of SHARP, our key idea is to align various images in a unified omnidirectional latent space. Thus, we propose UniSHARP, which performs implicit alignment in both feature and Gaussian spaces. Specifically, Gaussian primitives are arranged along rays and radial distances in a ray-based universal representation, while 2D semantic and 3D spatial features extracted from UniK3D-inspired encoders are jointly decoded to generate the complete Gaussian cloud. To comprehensively evaluate our method, we construct a benchmark covering diverse imaging systems across various scenes. The benchmark is further stratified by field of view (FoV) to enable fine-grained assessment of the universal monocular rendering task. Extensive experiments on the proposed benchmark demonstrate the effectiveness of UniSHARP, outperforming alternative methods by a large margin. The project page can be found at: https://insta360-research-team.github.io/Unisharp-website/