基于时空注意力链的快速4D网格生成
阅读原文· arxiv.org该研究提出一种无需训练的4D网格生成新方法,通过“时空注意力链”框架实现动态三维结构的快速重建。方法从锚定网格顶点出发,在潜在空间中追踪时间对应关系,避免了显式匹配的高计算成本。实验显示,新方法仅需9秒即可生成4D网格,速度比现有最优方法提升13倍且质量更优,还能处理长达16倍的视频序列而不降低质量。改进的对应关系使其在2D物体跟踪和4D跟踪任务中达到有竞争力的零样本性能,并首次在4D网格生成中实现了可靠的相机参数估计。
4D mesh generation has recently emerged as a powerful paradigm for recovering dynamic 3D structure from videos, but existing methods remain slow, computationally expensive, and difficult to scale to longer sequences. We introduce a training-free approach that accelerates 4D mesh generation while improving temporal correspondence quality. Our key observation is that temporal correspondences emerge inside a 4D backbone long before its generated meshes become visually accurate. We exploit this with a general framework we call Spatio-Temporal Attention Chain which propagates information across space and time. Starting from vertices on an anchor mesh, the chain maps vertices to latent tokens. It then follows temporal correspondences in latent space, and recovers frame-specific vertices through latent-to-vertex attention. This design avoids expensive explicit matching while preserving anchor mesh details and thereby improving dynamic mesh geometry and temporal consistency. Compared to state-of-the-art, our method generates a 4D mesh in 9 seconds, achieving a 13times speedup while producing higher-quality results. Moreover, our approach scales to videos up to 16times longer without degrading mesh quality. Beyond generation, the improved correspondences enable competitive zero-shot performance on two downstream tasks: 2D object tracking and 4D tracking. We further show that our framework enables reliable camera estimation, a capability not supported by prior 4D mesh generation methods.