HuggingFace Daily Papers(社区热门论文)
57AI 编辑部评分,满分 100

HelloWorld:让视频世界模型中的角色实现社交互动

2026-08-05 08:00· 1天前
AI 导读

HelloWorld 是一个支持用户与视频世界内角色进行社交互动的视频世界模型。用户只需按一次按钮,即可让屏幕中的角色转向镜头、挥手、点头或说出简短问候。该模型通过自蒸馏流程微调视频生成模型,并引入无需训练模块调节 DiT 交叉注意力掩码,以时间定位角色响应。

Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld

来源:HuggingFace Daily Papers(社区热门论文) · arxiv.org

HelloWorld:让视频世界模型中的角色实现社交互动

HuggingFace Daily Papers(社区热门论文)·2026-08-05 08:00·1天前
AI 导读

HelloWorld 是一个支持用户与视频世界内角色进行社交互动的视频世界模型。用户只需按一次按钮,即可让屏幕中的角色转向镜头、挥手、点头或说出简短问候。该模型通过自蒸馏流程微调视频生成模型,并引入无需训练模块调节 DiT 交叉注意力掩码,以时间定位角色响应。

原文 · 保持原样,未翻译

Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org