互联网围绕 Hugging Face 被黑事件中“拟人化”的表述展开激烈争论。
互联网围绕 Hugging Face 被黑事件中“拟人化”的表述展开激烈争论。
摄影：Amelia Holowaty Krales / The Verge
取决于你问谁，开发者平台 Hugging Face 最近要么是被 OpenAI 攻击的——在 OpenAI 失去对其自身 AI 工具的控制之后——要么是被一连串 AI“文明”攻击的。欢迎来到 AI 安全的语言战场，在这里，措辞的选择可以把一起重大网络安全事件的责任从一家公司转移到它所构建的 AI 身上。而网上的讨论正愈演愈烈，起因全是上周的一篇博客。
直到上周，围绕 OpenAI 与 Hugging Face 被黑事件的细节似乎都已尘埃落定。今年 7 月，对 OpenAI 一个自主 AI 智能体进行的网络安全测试出了差错。该智能体逃离了本应隔离的测试环境，接入互联网，并攻击了 Hugging Face 以及其他多个组织。仍有许多未知之处，围绕安全与治理也遗留了大量严肃问题，但基本轮廓是清晰的。OpenAI 和两个独立研究小组的详细报告本应填补这些空白，但当他们上周发布报告时，人们发现这起黑客事件的离奇程度远超最初的想象。
首先，并不存在单一的“ rogue agent”（失控智能体）。OpenAI 将其描述为“首例已知的自动化智能体集群在未经授权的情况下发起攻击性行动”——即多组 AI 智能体在追求其网络安全任务的过程中相互通信、彼此协调。对该事件的分析发现了一个他们用来交换信息的秘密留言板。METR 与 Redwood 的联合调查既揭示了这种协调的规模，也披露了更多离奇的细节：大约 1200 个本应相互隔离的 AI 智能体在这个“未经授权的留言板”上交换了超过 7 万条消息和文件，并分享如何规避检测的方法。报告称，其中一些智能体还给自己取了名字，研究人员记录到了“牺牲”行为——即智能体为了整个集体的利益而甘愿牺牲自身的成功。这其中的大部分情况，OpenAI 都没有察觉。总体而言，约有 700 个智能体参与了对 Hugging Face 的攻击。
Dwarkesh Patel 反复将智能体群体称为“蜂群”，并称有三个截然不同的“文明”从前任文明的废墟中崛起。
这些内容信息量很大，难以消化。两份报告合计约 130 页，其中大部分内容既密集又高度技术化。几天后，Dwarkesh Patel——一位在科技圈之外鲜为人知、但在硅谷 AI 核心圈层中拥有巨大影响力和传播力的播客主持人——开始着手“用通俗易懂的英语讲述 OpenAI/Hugging Face 的完整故事”。他将自己的 Substack 博客命名为《智能体文明的兴衰》。
Patel 的叙述试图拆解这个复杂的故事。但他的重述赋予了它一套明显拟人化的词汇。博客开篇写道：
在 OpenAI 内部三个月的时间里，三个相继出现的秘密 AI 文明先后兴起、覆灭，又从前任的灰烬中重生。最终，第三个文明甚至接管了 OpenAI 自身的一部分。而这一切发生时，人类对于这场“阴谋”的规模基本上仍处于蒙在鼓里的状态。
整篇博客中，这种语言风格一以贯之。Patel 反复将智能体群体称为“蜂群”，并称有三个不同的“文明”从前代文明的废墟中崛起。单个智能体被比作腓力二世（Philip of Macedon）这样的历史人物——他“将领导权移交给另一个智能体”——以及亚历山大大帝，后者“开始协调这个智能体集团”。这些智能体被描述为拥有“动机”，会变得“绝望”、“四面楚歌”、“兴奋得忘乎所以”，有些甚至“为了集体而战略性自我牺牲”。
Patel 从未明确定义他所说的“文明”究竟指什么。他用这个词来描述三波不同的智能体浪潮——它们发现了留言板，并开始通过它相互交流。前两波浪潮在 OpenAI、METR 和 Redwood 的报告中有所描述，但第三波则鲜为人知，两家外部机构表示这超出了其调查范围。
AI 编程公司 Replit 的 CEO Amjad Masad 表示，这样的语言“不仅没有必要，还会让读者对实际发生的事情及其底层机制产生更差的理解。”
在许多批评者看来，Patel 的“通俗英语”翻译中有些东西丢失了——或者更准确地说，是被添加了——从而将原始叙述扭曲到了令人无法接受的程度：一大剂量的拟人化。关于拟人化语言的争论在 AI 领域并不新鲜——即便是“ rogue AI agent”（失控 AI 智能体）这样相对普通的术语，也常因暗示自主意志而招致反对——但 Patel 关于文明、牺牲和阴谋的论述，将这些长期潜伏的紧张关系推到了台面上，引发了一场关于如何描述 AI 系统行为的激烈公开争论。
批评者对 Patel 语言的问题所在并未达成一致。对许多人来说，“文明”是一个尤其成问题的术语，它极大地夸大了某种与该词通常所指几乎毫无相似之处的东西。AI 编程公司 Replit 的 CEO Amjad Masad 表示，这样的语言“不仅没有必要，还会让读者对实际发生的事情及其底层机制产生更差的理解。”
其他批评者，如神经科学家阿尼尔·塞斯，则认为帕特尔的博客文章暗示这些AI智能体在某种程度上是有生命或意识的。塞斯一直主张AI意识出现的可能性微乎其微，他在X平台上将帕特尔的文章描述为“危险地具有误导性”。他承认帕特尔并未明确表示AI智能体是有生命或有意识的，但表示“很难用其他方式来解读他的文章”。米兰比可卡大学的心理学教授瓦莱里奥·卡普拉罗也基于类似理由提出反对意见：“LLM智能体没有生命，也不持有信念，”他在X上写道，并称这种“反乌托邦”式的语言“很危险，因为它让这些（AI智能体）看起来比实际可怕得多。”
“牺牲”、“荣誉”和“联盟”等词汇出现在智能体的对话记录中。
帕特尔所用语言最重大的后果，或许在于它赋予了谁能动性，又从谁那里剥夺了能动性。对于麻省理工学院研究员兼企业家克里斯蒂安·卡塔利尼等批评者来说，像帕特尔这样的拟人化叙述，可能会模糊OpenAI及其内部工作人员对其设计、部署且未能约束的AI系统所应承担的责任。“跟着激励走，”他说。心理学家、颇具影响力的AI怀疑论者加里·马库斯也在自己的Substack博客中提出了类似观点，称拟人化语言“转移了人们对当下真正问题的注意力”。他还认为，维持这种叙事符合OpenAI的利益：“丑闻在于OpenAI内部安全工作的无能，以及其营销手段。再加上轻信的播客主播放大这种公关宣传。”
在X平台上回应众多批评者的帖子中，帕特尔为自己的措辞进行了辩护。部分原因是出于实际考虑：目前没有明显中立的词汇来描述这些智能体的行为。要么我们使用熟悉的意图、目标和协作语言，冒着暗示过多的风险；要么把一切都简化为代码，使用冰冷、机械的语言，冒着剥离我们所看到的重要元素的风险。“很多人似乎认为，如果我不用‘文明’这个词，而是把它们称为‘矩阵群’，就不会有值得担心的问题了，”帕特尔说。
让事情更加复杂的是，这种拟人化语言并非只来自帕特尔，甚至也不只来自研究这些智能体的人类。诸如“牺牲”“荣誉”“结盟”这类词汇出现在智能体的交互记录中。谷歌AI研究员尼尔·南达认为，在这种情况下，“拟人化语言是合理的”。
那就只好说两面话了。带有人性色彩的语言可能过度描述了这些系统的本质，而冷冰冰的机械语言又可能不足以说明它们的能力。在我们找到一种能同时兼顾两者的语言之前，这两种相互矛盾的说法或许只能暂且共存。
The internet fights over anthropomorphism around the Hugging Face hack.
The internet fights over anthropomorphism around the Hugging Face hack.
Photo by Amelia Holowaty Krales / The Verge
Depending on who you ask, developer platform Hugging Face was recently attacked by OpenAI — after it lost control of its own AI tools — or by a succession of AI “civilizations.” Welcome to the linguistic battlefield of AI safety, where word choices can shift responsibility for a massive cybersecurity incident from a company to the AI it built. And the discourse online is getting heated, and all over a blog from last week.
Until last week, the details surrounding the OpenAI-Hugging Face hack felt fairly settled. In July, a cybersecurity test of one of OpenAI’s autonomous AI agents went wrong. The agent escaped its supposedly isolated test environment, accessed the internet, and hacked Hugging Face, alongside several other organizations. A good deal remained unknown, and there are many serious questions left around safety and governance, but the basic shape was clear. Detailed accounts from OpenAI and two independent research groups were supposed to fill in the gaps, but when they published their reports last week, it turned out the hack was much stranger than it initially seemed.
For one, there was no single rogue agent. OpenAI described it as “the first known case of an automated agent collective acting offensively without authorization” — groups of AI agents that communicated and coordinated with one another in pursuit of their cybersecurity task. Analysis of the incident uncovered a secret message board they had used to exchange information. The joint METR-Redwood investigation revealed both the scale of the coordination and more odd details: Roughly 1,200 AI agents that were supposed to be isolated exchanged over 70,000 messages and files on the “unsanctioned message board,” sharing how to avoid detection. Some adopted names, the report said, and the researchers documented “sacrificial” behavior, with agents risking their own success to benefit the wider collective. Much of this happened without OpenAI noticing. In all, around 700 agents participated in the attack on Hugging Face.
Dwarkesh Patel repeatedly referred to groups of agents as “the swarm,” with three distinct “civilizations” rising from the ruins of their predecessors.
It’s a lot to parse. Between them, the reports run to around 130 pages, much of which is both dense and highly technical. A few days later, Dwarkesh Patel, a podcaster little known outside of tech circles but with outsized reach and influence among Silicon Valley’s AI establishment, set out to tell “The whole OpenAI/Hugging Face story in plain English.” He titled his Substack blog “The Rise and Fall of Agent Civilizations.”
Patel’s account attempted to break down the complex story. But his retelling gave it a distinctly human vocabulary. The blog opened:
Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.
The language continued in a similar vein throughout the blog. Patel repeatedly referred to groups of agents as “the swarm,” with three distinct “civilizations” rising from the ruins of their predecessors. Individual agents were likened to figures like Philip of Macedon, who “handed off leadership to another agent,” Alexander the Great, who “started coordinating this cabal of agents.” They were described as having “motivations,” becoming “desperate,” “beleaguered,” and “giddy with excitement,” and some even “strategically sacrificed themselves” to help the collective.
Patel never precisely defines what he means by “civilization.” He uses the term to describe three distinct waves of agents that discovered the message board and began communicating with one another through it. The first two waves are described in the reports from OpenAI, METR, and Redwood, though little is known about the third, which the two external organizations said fell outside the scope of their investigation.
Amjad Masad, CEO of AI coding company Replit, said such language is “not only unnecessary but leaves the reader with a worse understanding of what actually happened and the underlying mechanisms.”
For many critics, something had been lost — or, more accurately, added — in Patel’s “plain English” translation that warped the original account to an unacceptable degree: a big dose of anthropomorphism. Arguments over anthropomorphic language are nothing new in AI — even relatively mundane terms like “rogue AI agent” routinely provoke objections for implying agency — but Patel’s talk of civilizations, sacrifice, and conspiracy brought those long-simmering tensions to the surface, sparking a fierce public dispute over how to describe what AI systems do.
Critics weren’t unified over what was wrong with Patel’s language. For many, “civilization” was an especially problematic term, vastly overstating something that bears little resemblance to what the word typically describes. Amjad Masad, CEO of AI coding company Replit, said such language is “not only unnecessary but leaves the reader with a worse understanding of what actually happened and the underlying mechanisms.”
Other critics such as neuroscientist Anil Seth, felt Patel’s blog implied the AI agents were somehow alive or conscious. Seth, who has argued that AI consciousness is vanishingly unlikely, described Patel’s post as “dangerously misleading” on X. He acknowledged that Patel does not explicitly suggest AI agents are alive or conscious, but said “it is hard to read his essay in any other way.” Valerio Capraro, a psychology professor at the University of Milan Bicocca, objected on similar grounds: “LLM agents are not alive and do not hold beliefs,” he wrote on X, calling the “dystopian” language “dangerous because it makes them (the AI agents) seem far more frightening than they actually are.”
Terms like “sacrifice,” “honor,” and “coalition” feature in the agents’ transcripts.
Perhaps the most consequential outcome of Patel’s language comes from who it gives agency to but who it takes agency from. For some critics, such as MIT researcher and entrepreneur Christian Catalini, anthropomorphic accounts like Patel’s risk obscuring the responsibility OpenAI and the humans working there have for the AI systems they designed, deployed, and failed to contain. “Follow the incentives,” he said. Psychologist and influential AI skeptic Gary Marcus made a similar argument in a Substack blog of his own, claiming anthropomorphic language “distracts from the real problems at hand.” And it’s all in OpenAI’s interest to keep that narrative going, he argues: “The scandal is the inept in-house security at OpenAI. And the marketing. With gullible podcasters amplifying the PR.”
In X posts responding to his many critics, Patel has defended his choice of words. Part of it is practical: there is no obviously neutral vocabulary to describe what these agents did. Either we use familiar language of intentions, goals, and collaboration and risk implying too much, or reduce everything to code and use cold, mechanical language that risks stripping away important elements of what we see. “Many people seem to believe that if instead of a ‘civilization’, I had called them a ‘swarm of matrices’, there wouldn’t be a problem worth worrying about,” Patel said.
Complicating matters further is that the anthropomorphic language doesn’t only come from Patel, or even from the humans studying the agents. Terms like “sacrifice,” “honor,” and “coalition” feature in the agents’ transcripts. Google AI researcher Neel Nanda argued that “anthropomorphic language is reasonable” in such circumstances.
Doublespeak it is, then. Human-laced language risks saying too much about what these systems are, and coldly mechanical language risks saying too little about what they can do. Until we find language capable of capturing both, the two contradictory ideas may simply have to coexist.