Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.
SkillGate:长程智能体中的策略内技能选择训练
AI 导读
SkillGate 解决了智能体策略在回合中途选择技能时缺乏训练信号的问题,通过将 token 支持划分为结果信用与动作局部优势两个不相交通道,消除了选择器信用匮乏。在 16 候选技能下,该方法将 9B 策略的试验成功率从 40.8% 提升至 53.2%,并将误导候选暴露减少三分之二。
HuggingFace Daily Papers(社区热门论文)
44
AI 编辑部评分,满分 100SkillGate:长程智能体中的策略内技能选择训练
SkillGate 解决了智能体策略在回合中途选择技能时缺乏训练信号的问题,通过将 token 支持划分为结果信用与动作局部优势两个不相交通道,消除了选择器信用匮乏。在 16 候选技能下,该方法将 9B 策略的试验成功率从 40.8% 提升至 53.2%,并将误导候选暴露减少三分之二。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org