HuggingFace Daily Papers(社区热门论文)
61AI 编辑部评分,满分 100

ToolHazard:面向LLM智能体的可扩展对抗环境合成框架

2026-08-12 08:00· 1天前
AI 导读

ToolHazard提出可扩展的对抗环境合成框架,通过环境模拟器、攻击者智能体和用户模拟器生成可执行有状态环境,自动发现注入点并构造环境特定载荷与长程任务。基于该框架构建的ToolHazard-Bench在复杂工作流与多样攻击下暴露智能体显著脆弱性,且注入时机与位置影响攻击效果。

Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose ToolHazard, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build ToolHazard-Bench for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.

来源:HuggingFace Daily Papers(社区热门论文) · arxiv.org

ToolHazard:面向LLM智能体的可扩展对抗环境合成框架

HuggingFace Daily Papers(社区热门论文)·2026-08-12 08:00·1天前
AI 导读

ToolHazard提出可扩展的对抗环境合成框架,通过环境模拟器、攻击者智能体和用户模拟器生成可执行有状态环境,自动发现注入点并构造环境特定载荷与长程任务。基于该框架构建的ToolHazard-Bench在复杂工作流与多样攻击下暴露智能体显著脆弱性,且注入时机与位置影响攻击效果。

原文 · 保持原样,未翻译

Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose ToolHazard, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build ToolHazard-Bench for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org