CityRAG:通过空间锚定视频生成步入城市
阅读原文· arxiv.orgCityRAG 是一种新型视频生成模型,通过利用大规模地理注册数据作为上下文,将生成内容锚定到真实物理场景。该模型采用时间未对齐的训练数据,学会从瞬态属性中语义解耦底层场景,从而在保持复杂运动和外观变化先验的同时,实现真实世界重建。实验表明,该系统可生成数分钟长的连贯视频序列,在数千帧内保持天气和光照条件的一致性,支持闭环导航和复杂轨迹重建真实地理环境。
We address the problem of generating a 3D-consistent, navigable environment that is spatially grounded: a simulation of a real location. Existing video generative models can produce a plausible sequence that is consistent with a text (T2V) or image (I2V) prompt. However, the capability to reconstruct the real world under arbitrary weather conditions and dynamic object configurations is essential for downstream applications including autonomous driving and robotics simulation. To this end, we present CityRAG, a video generative model that leverages large corpora of geo-registered data as context to ground generation to the physical scene, while maintaining learned priors for complex motion and appearance changes. CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes. Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.