通过可形变物体先验实现相机空间中的类别级3D对应关系
阅读原文· arxiv.org为解决机器人与AR/VR领域中单张图像理解3D物体时语义粒度不足的问题,研究提出了通过学习共享的可形变物体先验,无需显式对应监督即可在相机空间中获得类别级3D对应。为此,团队引入了首个大规模基准测试HouseCorr3D,包含178k图像、50个家居物体类别及280个实例的CAD模型3D关键点标注,并提供了非模态对应标签与对称性标注。同时提出的Morpheus方法,通过解耦标准形状、形变与物体姿态来学习该先验,从而隐式涌现语义对应的3D理解,并在该基准上达到了新的SOTA。数据与代码已开源。
Understanding 3D objects from images is fundamental to robotics and AR/VR applications. While recent work has made progress in category-level pose estimation, current representations fail to capture the fine-grained semantics needed for reasoning about object parts, functions, and interactions. In this work, we study category-level 3D correspondence in camera space -- predicting, from a single image, 3D locations that remain consistent across instances within a category -- and show that it can emerge without explicit correspondence supervision by learning a shared morphable object prior. To enable research in this direction, we introduce HouseCorr3D, the first large-scale benchmark for monocular category-level 3D correspondence with 178k images across 50 household object categories, 280 unique instances, and 3D keypoint annotations directly on CAD models. Crucially, HouseCorr3D provides amodal correspondence labels for occluded regions and explicit symmetry annotations, addressing key limitations of existing datasets. We further propose Morpheus, a method that learns morphable category-level shape priors by disentangling canonical shape, deformation, and object pose. Through this shared canonical grounding, semantically meaningful 3D correspondences in camera space emerge implicitly. These emerging 3D correspondences set a new state of the art on HouseCorr3D, demonstrating that semantic 3D object understanding can arise without direct correspondence supervision. Data and code are publicly available at https://github.com/GenIntel/HouseCorr3D.