LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 percentage points.
StepGuard:以可扩展监督与安全-效用平衡实现步骤级护栏
AI 导读
StepGuard 是一种步骤级护栏模型,可在工具调用前审查智能体动作并审计完整轨迹。其训练采用自动数据引擎 StepGen 生成同语境不同动作的安全/不安全轨迹,并以 Balance-GRPO 动态平衡安全与不安全动作的学习。
HuggingFace Daily Papers(社区热门论文)
48
AI 编辑部评分,满分 100StepGuard:以可扩展监督与安全-效用平衡实现步骤级护栏
StepGuard 是一种步骤级护栏模型,可在工具调用前审查智能体动作并审计完整轨迹。其训练采用自动数据引擎 StepGen 生成同语境不同动作的安全/不安全轨迹,并以 Balance-GRPO 动态平衡安全与不安全动作的学习。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org