# WorldReward 提出面向相机条件世界模型的 VLM 奖励建模方法

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-09-03 08:00
- AIHOT 分数：40
- AIHOT 链接：https://aihot.virxact.com/items/cmtmeb7fn00xarotxful3ecm6
- 原文链接：https://arxiv.org/abs/2609.03952

## AI 摘要

WorldReward 是一个基于 VLM 的成对偏好奖励模型，为相机条件世界模型统一评估动作一致性与视觉质量。方法将视频分解为动作对齐的块并投票聚合，配套发布人工标注的 WorldReward-Bench，在三个维度上分别超过 GPT-5.5 约 3.42、1.45 和 3.56 个百分点，用于 HY-WorldPlay 1.5 的 RL 后训练可同时提升动作执行与视觉质量。

## 正文

Camera-conditioned world models generate interactive videos in which commanded actions should induce the expected scene changes while appearance, geometry, and temporal dynamics remain coherent. Existing rewards assess these requirements separately: geometry-based rewards estimate trajectory execution but cannot judge the visual quality of the executed motion, whereas image-based rewards measure frame quality without capturing action execution or temporal dynamics. We posit that a vision-language model (VLM) offers a shared reasoning space for relating actions to their visual outcomes. However, judging a complete long video against its full action sequence creates a lengthy, noisy context in which short-lived local action evidence can be missed or diluted. We present WorldReward, a VLM-based pairwise preference reward model that unifies action-consistency and visual-quality evaluation for camera-conditioned world models. WorldReward decomposes paired videos into action-aligned chunks, organizes each chunk into structured visual evidence, and aggregates chunk-level decisions by voting into separate video-level action and visual-quality preferences. To train it, we construct a large-scale reasoning-augmented preference dataset using structured judgments generated by a frontier VLM and refined through tool-based agent auditing and targeted human review. We further introduce WorldReward-Bench, a human-annotated benchmark measuring reward-model agreement with human preferences across action consistency, appearance quality, and motion quality. WorldReward achieves the highest agreement on all three dimensions, exceeding GPT-5.5 by 3.42, 1.45, and 3.56 percentage points, respectively. When used for RL post-training of HY-WorldPlay 1.5, it consistently improves both action execution and visual quality across short- to long-term horizons.