Goku:面向指令视频编辑的百万级通用数据集与基准
阅读原文· arxiv.orgGoku是一个包含200万高质量视频编辑对的百万级数据集,首次将基于指令的视频编辑从单一外观扩展至多任务和结构操控(如主体运动控制)。研究者设计了分解式数据合成流水线与渐进式过滤系统以解决复杂编辑的数据合成难题。基于该数据集训练的Goku-Edit模型采用MLLM作为文本编码器,并采用解耦双分支架构:专用掩码分支处理结构控制,主分支负责外观渲染。配套基准Goku-Bench包含1000个人工验证测试用例和7项新增编辑指标,Goku-Edit的指令遵循能力较其他开源模型提升高达+8%。
Existing instruction-based video editing datasets commonly focus on single-task appearance editing, failing to meet the complex creative demands of real-world scenarios. To bridge this gap, we present Goku, a large-scale dataset featuring 2 million high-quality, instruction-aligned video editing pairs, which is the first to extend task boundaries from basic appearance editing to multi-task and structural manipulations(e.g., precise control of subject movement). To tackle the data synthesis challenges inherent in these complex tasks, we design an efficient data synthesis pipeline that decomposes complex edits into controllable sub-problems and introduce a progressive filtering system for data reliability throughout the whole process. Furthermore, we explore the optimal network structures on Goku, and propose Goku-Edit. To deeply comprehend complex editing instructions, Goku-Edit leverages an MLLM as its text encoder and adopts a decoupled dual-branch design: a dedicated mask branch handles structural control, freeing the main branch for appearance rendering. A comprehensive video editing benchmark, Goku-Bench, is also proposed with 1,000 human-verified test cases and 7 novel editing-specific metrics. Evaluated on Goku-Bench, Goku-Edit obtains up to +8% improvement on other open-source models in terms of instruction following.