ReWorld:具备长时记忆的交互式世界模型

HuggingFace Daily Papers(社区热门论文)·2026-08-24 08:00·1天前
AI 导读

ReWorld 将控制与记忆分离训练并在推理时加以约束,通过混合逐头注意力窗口和随机头路由,让少量全局头关注全部历史,同时以姿态索引地标库支撑有界 KV 缓存。在覆盖动作跟随、长时回忆和视频质量的三轴评测中,ReWorld 对六个近期交互式世界模型取得最佳控制保真度(11.95° 旋转误差)和最佳生成质量,并支持 704x1280 实时视频流。

HuggingFace Daily Papers(社区热门论文)
37AI 编辑部评分,满分 100

ReWorld:具备长时记忆的交互式世界模型

2026-08-24 08:00· 1天前
AI 导读

ReWorld 将控制与记忆分离训练并在推理时加以约束,通过混合逐头注意力窗口和随机头路由,让少量全局头关注全部历史,同时以姿态索引地标库支撑有界 KV 缓存。在覆盖动作跟随、长时回忆和视频质量的三轴评测中,ReWorld 对六个近期交互式世界模型取得最佳控制保真度(11.95° 旋转误差)和最佳生成质量,并支持 704x1280 实时视频流。

An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounded one. ReWorld separates the two during training and bounds them at inference. Mixed per-head attention windows confine most heads to the recent past while a small set of global heads attends over the entire history, and random head routing keeps either capability from binding to particular heads; random chunk dropping makes sparse histories in-distribution. At inference the whole past lives under a fixed budget: a bounded KV cache backed by a pose-indexed landmark bank, from which the model retrieves the landmarks nearest the current pose. A metric-scale-aligned data engine places eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale, so the same key press moves the camera the same distance in every source, and palindrome trajectories supply the revisit evidence that memory training needs. Distribution-matching distillation confined to a LoRA adapter then compresses sampling to four steps: one backbone serves both a high-fidelity multi-step mode and a real-time interactive one, streaming 704x1280 video across photorealistic, game-style, and stylized worlds. Under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity (11.95^circ rotation error and the best camera-motion consistency) and the best generation quality; and on minute-long out-and-back rollouts (64,s, 384 latents), its fixed 12-chunk cache still regenerates the starting view -- at rollout lengths where a sliding window has long evicted the evidence and full-KV attention runs out of memory.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org