SkillGate:长程智能体中的策略内技能选择训练

HuggingFace Daily Papers(社区热门论文)·2026-08-19 08:00·16天前
AI 导读

SkillGate 解决了智能体策略在回合中途选择技能时缺乏训练信号的问题,通过将 token 支持划分为结果信用与动作局部优势两个不相交通道,消除了选择器信用匮乏。在 16 候选技能下,该方法将 9B 策略的试验成功率从 40.8% 提升至 53.2%,并将误导候选暴露减少三分之二。

HuggingFace Daily Papers(社区热门论文)
44AI 编辑部评分,满分 100

SkillGate:长程智能体中的策略内技能选择训练

2026-08-19 08:00· 16天前
AI 导读

SkillGate 解决了智能体策略在回合中途选择技能时缺乏训练信号的问题,通过将 token 支持划分为结果信用与动作局部优势两个不相交通道,消除了选择器信用匮乏。在 16 候选技能下,该方法将 9B 策略的试验成功率从 40.8% 提升至 53.2%,并将误导候选暴露减少三分之二。

Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org