Helix4D:复杂动态网格生成
阅读原文· arxiv.orgHelix4D是一个动态网格生成框架,旨在解决现有方法在处理复杂拓扑、透明材质和薄结构等场景时的不足。该框架继承了Trellis2的强大表示能力,将其从图像生成3D扩展到基于视频的4D生成。为兼顾信息共享与预训练质量,它采用了滑动窗口跨帧注意力并锚定第一帧。同时,其设计的4D时序编码复用了低频空间RoPE频段来表示时间维度,无需额外参数。实验在ActionBench和自建复杂数据集上验证了该方法生成高质量动态网格的有效性。
Current video-to-4D methods struggle with complex topology changes, transparent materials, thin structures, and inner surfaces. We present Helix4D, a dynamic mesh generation framework by inheriting the expressive representation of Trellis2, adapting it from image-to-3D to video-conditioned 4D generation. Our design arises from two key questions: (a) how to enable Trellis2's frame-local attention to share information across frames while preserving its pretrained quality on rare cases such as transparent objects and inner surfaces, and (b) how to inject temporal information into a purely 3D positional encoding without breaking pretrained capabilities. We address (a) with a sliding-window cross-frame attention and anchor on the first frame. The first frame is generated by the base Trellis2 model and injected into our model, letting it inherit Trellis2's quality in rare cases through cross-frame attention. We address (b) with a 4D temporal encoding that repurposes redundant low-frequency spatial RoPE bands for time, extending the encoding from 3D with no additional parameters. Extensive experiments show the effectiveness of Helix4D for high-quality dynamic mesh generation on ActionBench and our own challenging complex dynamics set.