AutoRubric-T2I:用于文本-图像对齐的基于规则的鲁棒奖励模型
阅读原文· arxiv.org本研究针对现有文本-图像生成模型奖励模型训练成本高、评估标准不透明的问题,提出了首个自动评规学习框架AutoRubric-T2I。该框架能够自动合成并筛选明确的评分规则,以指导视觉语言模型评判者。其核心方法在于从偏好对中合成为候选规则,并通过带L1正则化的精炼器筛选出最具区分性的规则。实验表明,该框架仅需不到0.01%的标注数据即可生成高质量且可解释的奖励信号,在多个基准测试中性能优于强基线,并能有效提升下游生成任务的质量。
Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment and perceptual quality. Existing reward models are commonly trained as Bradley-Terry (BT) preference models on large-scale human preference corpora, making them costly to train, difficult to adapt, and opaque in their evaluation criteria. Meanwhile, Vision-Language Model (VLM) judges can provide more fine-grained assessments through textual rubrics, but their manually designed or heuristically generated scoring rules may fail to reliably reflect human preferences. In this paper, we propose AutoRubric-T2I, the first rubric learning framework in T2I that automatically synthesizes and selects explicit rubrics for guiding VLM judges. AutoRubric-T2I first synthesizes reasoning traces from preference pairs into candidate rubrics, then uses a VLM judge to score paired images under each rubric, producing pairwise rubric-score differences for preference learning. To remove noisy and redundant rules, we further employ a ell_1-Regularized Logistic Regression Refiner, which selects the Top-N most discriminative rubrics. Extensive evaluations show that AutoRubric-T2I produces high-quality, interpretable reward signals using less than 0.01% of the annotated preference data, substantially reducing the need for large-scale reward-model training. On image reward benchmarks such as MMRB2, AutoRubric-T2I outperforms strong reward model baselines. We further validate AutoRubric-T2I as an RL reward on downstream T2I tasks, including TIIF and UniGenBench++, where it improves generation quality over scalar reward models using the Flow-GRPO pipeline on diffusion models.