# VA-Judger：面向视频-音频联合生成的人类偏好奖励模型

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-19 08:00
- AIHOT 分数：52
- AIHOT 链接：https://aihot.virxact.com/items/cmt290kr90ktrroovd1h0fcvc
- 原文链接：https://arxiv.org/abs/2608.18607

## AI 摘要

VA-Judger 提出一种面向视频-音频联合生成的链式思维全模态奖励模型，并构建了包含 9K 提示词和 10.3K 细粒度成对比较的大规模人类偏好数据集 VAPref-10K。该模型通过分维度强化学习分解人类反馈，在域内和域外评测中均优于现有指标基线，用于后训练可显著提升生成质量。

## 正文

Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual quality dimensions, including audio quality, visual fidelity, and synchronization. However, these metrics evaluate perceptual dimensions separately and fail to capture the overall semantic and temporal coherence among the text prompt, video, and audio that shapes human preferences. Optimizing models against these metrics encourages reward hacking, generating video-audio content that achieves high scores on these metrics yet appears incoherent or unfaithful to human viewers. To address this problem, we first construct a large-scale human-preference dataset VAPref-10K for joint video-audio generation, comprising 9K prompts and 10.3K fine-grained paired comparisons from open-source generation models. We also introduce the VA-Judger-Bench benchmark with both in-domain and out-of-domain model comparisons to evaluate whether reward models truly align with human preferences. We further propose VA-Judger, a chain-of-thought omni-reward model for joint video-audio generation. In particular, VA-Judger first learns from pairs with clear quality gaps to establish structured output and coarse preference discrimination, then distills reliable preference explanations for harder near-quality comparisons via rejection sampling verified against human annotations, and finally performs dimension-wise reinforcement learning that decomposes human feedback into individual quality dimensions for denser reward signals than a single binary preference label. Experiments show that VA-Judger outperforms metric baselines in predicting human preferences on both in-domain and out-of-domain evaluations. Using its human-aligned rewards for post-training audio-video generation model also yields significant improvements in generation quality.