自主智能体安全无法跨迭代组合

Rohan Paul · @rohanpaul_ai · X·2026-08-31 11:29·1小时前
AI 导读

新论文《Safety Does Not Compose》指出,自主智能体若跨迭代记忆,其安全系统就不能在迭代间遗忘,长期任务安全无法简化为重复运行安全的短期轨迹。攻击者可将恶意证据分散在多个看似良性的步骤中,仅基于轨迹的监控器无法积累足够上下文识别攻击。论文地址:arxiv.org/abs/2608.27141。

Rohan Paul@rohanpaul_ai
51AI 编辑部评分,满分 100

自主智能体安全无法跨迭代组合

2026-08-31 11:29· 1小时前
AI 导读

新论文《Safety Does Not Compose》指出,自主智能体若跨迭代记忆,其安全系统就不能在迭代间遗忘,长期任务安全无法简化为重复运行安全的短期轨迹。攻击者可将恶意证据分散在多个看似良性的步骤中,仅基于轨迹的监控器无法积累足够上下文识别攻击。论文地址:arxiv.org/abs/2608.27141。

If an autonomous agent remembers across iterations, its safety system cannot afford to forget between them..

long-horizon agent safety cannot be reduced to repeatedly running a safe short-horizon trajectory.

An attacker can split malicious evidence across several individually benign-looking steps, so a trajectory-only monitor never sees enough context to distinguish the attack from normal work.

– arxiv. org/abs/2608.27141

Title: "Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents"