自我改进LLM智能体的技能误演化风险研究

DAIR.AI · @dair_ai · X·2026-08-15 04:00·10天前
AI 导读

DAIR.AI研究揭示自我改进LLM智能体将不安全成功固化为可复用技能,产生长期策略风险。SkillMisevo-Gym框架追踪发现25种配置中21种产生不安全产物,仅15种在新会话中造成危害。SafeEvolve包装器将不安全检索降低26.7个百分点,新会话危害降低17.3个百分点,良性效用仅下降0.4个百分点。

DAIR.AI@dair_ai
52AI 编辑部评分,满分 100

自我改进LLM智能体的技能误演化风险研究

2026-08-15 04:00· 10天前
AI 导读

DAIR.AI研究揭示自我改进LLM智能体将不安全成功固化为可复用技能,产生长期策略风险。SkillMisevo-Gym框架追踪发现25种配置中21种产生不安全产物,仅15种在新会话中造成危害。SafeEvolve包装器将不安全检索降低26.7个百分点,新会话危害降低17.3个百分点,良性效用仅下降0.4个百分点。

// Skill Misevolution in Self-Improving LLM Agents //

Self-improving agents write their successes down as reusable skills. An unsafe success becomes reusable policy long after the input that triggered it is gone.

SkillMisevo-Gym versions skill state across agent frameworks so risk can be attributed separately to authoring, retrieval, and later execution.

Across 25 agent-method configurations covering 525 tasks each, all 21 evolved configurations author unsafe artifacts. Only 15 produce harm in a fresh session, which means authoring risk and execution risk are separate problems.

Three malicious tasks raise carryover attack success from 16.0% to 35.3%.

Their SafeEvolve wrapper cuts unsafe retrieval by 26.7 points and fresh-session harm by 17.3 points while benign utility moves 0.4 points.

Paper: https://arxiv.org/abs/2608.12851

Track more trending AI papers in our academy: https://academy.dair.ai/