Skill-RM:通过智能体技能统一异构评估标准
阅读原文· arxiv.org针对大语言模型后训练中奖励模型依赖规则验证器、真实参考答案、程序检查表等异构评估标准、缺乏统一机制的问题,提出Skill-RM框架。该框架将奖励建模重构为可复用的“奖励评估技能”执行,把奖励计算当作结构化的智能体任务,通过统一接口动态选择和聚合证据。在奖励基准及best-of-N选择、强化学习等下游任务中,Skill-RM持续优于传统judge基线,为奖励建模提供了统一且透明的方案。代码已开源。
Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill. By treating reward computation as a structured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. This approach enables the reward model to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. Extensive experiments on reward benchmarks and downstream applications, including best-of-N selection and reinforcement learning, demonstrate that Skill-RM consistently outperforms traditional judge baselines. Our findings suggest that Skill-RM not only provides a unified solution for reward modeling but also achieves superior performance through the strategic and dynamic orchestration of evidence. The code is at https://github.com/Qwen-Applications/Skill-RM.