Frontier agentic systems powered by large language models (LLMs) exhibit human-like patterns of cognition. As these systems become deeply integrated across different domains, their cognitive engagement raises critical concerns for human society that remain insufficiently studied. To address this gap, we systematically analyze risks induced by expanding cognitive capabilities, following a three-level framework defined by their cognitive scope, from physical cognition to social cognition, and finally to self-referential cognition. We study their potential risks to human agency, autonomy, and control capability, corresponding to each cognitive level. We finally propose strategies to mitigate these risks and enhance the controllability of agentic AI systems, ensuring their long-term safe development.
理解智能体 AI 系统中的认知诱发风险
AI 导读
一项研究系统分析了由大语言模型驱动的前沿智能体系统因认知能力扩展而引发的风险,按认知范围分为物理认知、社会认知到自我指涉认知三个层级。研究对应考察了这些风险对人类能动性、自主性和控制能力的影响,并提出缓解策略以增强智能体 AI 系统的可控性,保障其长期安全发展。
HuggingFace Daily Papers(社区热门论文)
34
AI 编辑部评分,满分 100理解智能体 AI 系统中的认知诱发风险
一项研究系统分析了由大语言模型驱动的前沿智能体系统因认知能力扩展而引发的风险,按认知范围分为物理认知、社会认知到自我指涉认知三个层级。研究对应考察了这些风险对人类能动性、自主性和控制能力的影响,并提出缓解策略以增强智能体 AI 系统的可控性,保障其长期安全发展。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org