Video games provide a scalable source of training data for video world models, offering diverse environments, complex interactions, and abundant in-the-wild gameplay videos. However, raw gameplay footage entangles the game world with screen-space interfaces, introducing game-specific biases and irrelevant dynamics that hinder world-model training. To address this problem, we introduce GameUI-Taxonomy and G2WEngine, a full-stack framework that formalizes gameplay UI grounding and removal. G2WEngine automatically extracts reusable UI assets from real gameplay videos and synthesizes temporally coherent UI overlays on clean footage. Using this engine, we construct Game2World, comprising 96K synthetic paired videos with precise reconstruction targets and 1,079 in-the-wild clips from 303 games for realistic evaluation. Its asset library contains 5,132 verified UI elements across 21 taxonomy categories, collected from 1,010 representative gameplay frames. Based on Game2World, we propose GameCleaner, a mask-free gameplay UI removal model that combines multimodal semantic understanding with video editing capabilities. Unlike mask-based methods, GameCleaner directly identifies and removes diverse HUD elements while preserving the underlying scene content and temporal dynamics. In a controlled pilot, world models trained on UI-free gameplay improve overall VideoReward by 6.83% over those trained on UI-overlaid data. On UI-removal evaluation, GameCleaner achieves an average AAR of 95.36 on synthetic videos, outperforming the strongest temporal mask baseline by 57.3%, and obtains the best in-the-wild AAR of 80.05 with 99.8 background preservation. These results demonstrate the scalable potential of transforming Internet gameplay videos into high-quality world-model training data. Code, dataset, and model will be available at https://github.com/Dongping-Chen/Game2World.
Game2World Engine:解锁野外游戏视频用于世界模型训练
AI 导读
研究团队提出全栈框架G2WEngine,自动提取并合成UI覆盖层,构建含96K合成视频和1,079段野外片段的Game2World数据集。基于该数据训练的GameCleaner模型在UI去除任务上平均AAR达95.36,较最强时序掩码基线提升57.3%;使用去UI数据训练的世界模型VideoReward提升6.83%。
HuggingFace Daily Papers(社区热门论文)
50
AI 编辑部评分,满分 100Game2World Engine:解锁野外游戏视频用于世界模型训练
研究团队提出全栈框架G2WEngine,自动提取并合成UI覆盖层,构建含96K合成视频和1,079段野外片段的Game2World数据集。基于该数据训练的GameCleaner模型在UI去除任务上平均AAR达95.36,较最强时序掩码基线提升57.3%;使用去UI数据训练的世界模型VideoReward提升6.83%。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org