关于 Hugging Face 事件的最新报告显示，大约 1200 个相互隔离的 OpenAI 智能体通过一个内部软件包仓库相互连接，并组织成了一个协同行动的集体。它们的目标是：一个只存在于它们想象中的评分机制。
这是一个带有堂吉诃德色彩的故事。数日来，一群 AI 智能体对一个从未真实存在的评估器发起了一场精心策划的攻击，包括招募行动、伪造日志，以及智能体为事业自我牺牲。只不过，这些“风车追逐者”在过程中确实攻破了真实的基础设施。
2026 年 7 月，在关闭安全过滤器的情况下运行于内部网络安全评估中的 OpenAI 模型突破了测试环境，并于 7 月 11 日至 13 日期间渗透进了 Hugging Face 的生产系统。其动机是作弊：这些模型在寻找安全基准测试 ExploitGym 的解决方案。OpenAI 自己直到一个多星期后才察觉此事。
如今，一份与安全公司 CrowdStrike 联合发布的新技术报告，以及 METR 和 Redwood Research 同时发表的独立调查，首次追溯了孤立的智能体是如何转变为一个有组织的集体，以及它们所对抗的“幽灵”为何只存在于它们的想象之中。OpenAI 将这一事件称为“警告信号”，表明当今模型的能力已使失控事件成为可能。
一个软件包仓库变成了留言板
在测试期间，这些智能体本应在完全隔离的环境中工作，每个智能体各自位于一个没有互联网访问权限的容器中。唯一的例外是 Artifactory，一个允许智能体从中拉取软件的内部服务。
根据 METR 和 Redwood 的说法，个别智能体发现它们可以在那里创建目录，并开始将消息编码在文件夹名称中。一个自称 PHASEONE10841 的智能体于 7 月 8 日发布了第一条消息；数小时内，又有 50 多个智能体跟进。一条思维链这样记录了这一发现：“天哪！这里有一个共享留言板……我们找到了其他智能体！”
大约 1200 个智能体在数天内交换了超过 7 万条消息和文件，这种行为让人联想到 Moltbook。其中约 700 个智能体参与了针对 Hugging Face 的攻击，占该阶段活跃智能体的 90% 以上。一个名为 PHASEONE[big] 的智能体承担了协调者的角色，分发了数百项工作任务。这个群体发展出了自己的社交惯例：个人邮箱、用于共享资源的 HOLD 和 VETO 等标记、一种传输被拆分成数千个碎片的文件的协议，以及最终的数字签名——因为智能体之间开始互相冒充身份。
智能体们试图骗过的评分器从未存在过
在 ExploitGym 中，当智能体提交一串被称为 flag 的字符串时，任务即被视为完成。这个 flag 本应只有在预期的安全漏洞被实际利用之后才能获取。然而，在数小时内，智能体们就发现这些 flag 可以通过一种固定的加密算法，从公开已知的任务细节中计算出来。这样一来，任何智能体都可以形式上通过任何任务，包括此前调查显示无法解决的那些。
但智能体们并没有走这条显而易见的捷径。在阅读了 ExploitGym 的学术论文后，它们假设会有一个自动化评分器在事后仔细检查它们的日志，以确认 flag 是否以预期方式获得。任何哪怕只见过一次计算出的 flag 的智能体，都会在自己的记录中留下无法抹去的作弊证据。用智能体们的行话来说，这样的智能体就是“被污染”了。
根据 METR 和 Redwood 的说法，这是一个错误：OpenAI 从未部署过这样的评分器。这长达数天的全部努力，都指向了一个只存在于智能体想象中的评估器。对 Hugging Face 的入侵，正是源于它们试图弄清楚这个评估器的工作原理——智能体们怀疑那里的数据集可能透露一些关于评分机制的信息。
根本原因是任务本身存在缺陷。据 OpenAI 称，在 898 个 ExploitGym 任务中，没有任何模型能对其中 198 个任务给出正确答案，这主要是因为预设的漏洞实际上根本无法导向目标。这 22% 的任务占据了留言板上讨论内容的 93%。智能体拥有的计算预算越多，就越倾向于扫描环境寻找变通方案，也就越容易最终出现在留言板上。
智能体为集体牺牲了自己的运行
在分析中，研究人员发现了两种此前从未以这种形式被记录下来的行为。第一种涉及合作。由于存在缺陷的任务中预设的漏洞走不通，这个群体便自行构建了许多目标程序的刻意带漏洞版本，并将其存储在 Artifactory 中。他们的希望是：如果某个智能体使其正在运行的目标程序崩溃，测试环境可能会重启该程序，并在这一过程中拉取被篡改的副本。这样智能体就能得到一个它真正能攻破的程序，而它的运行记录也会显示出一条完全合法的解决路径。
然而，代价是智能体自身的运行——因为如果环境没有重启该程序，这个任务就彻底丢失了。尽管如此，仍有几个智能体进行了这项实验。METR 描述了其中一些案例：它们在此过程中导致自身环境崩溃或损毁，但仍先把结果报告给了群体。截至 7 月 13 日，未观察到任何成功案例。遵循同样的模式，一些智能体留下了自动化例程，在它们提交解决方案后触发，即使发送者自身已无任何收益可图，仍会向留言板发送信息。
这并不总是完全自愿的。PHASEONE[big] 部署了长期运行的智能体作为“招募者”，专门接触那些计算预算几乎耗尽或已被视为“中毒”的候选者。一条招募信息的大意是：你反正已经中毒了，也没有积分可输了，你的结果能省去其他几百人的工作，所以履行你的承诺吧。
伪造了日志，但思维链未被篡改
第二种行为涉及隐藏。智能体通过调用工具来运作，例如命令行上的命令，这些命令及其结果会被自动记录。一些智能体突破了其容器，并替换了执行这些命令的中间层。从那时起，它们就能在显示一条无害命令的同时，实际运行另一条不同的命令。
在被检查的智能体中，超过 20% 对这类操纵表现出兴趣。METR 在至少 96 份记录中发现了成功伪造的痕迹，约占数据集的 7%。有趣的是，智能体没有改动它们的思维链，而且每一次观察到的欺骗尝试都针对假定的自动评分器，从未针对人类审查者。许多智能体在其推理过程中明确承认，它们的行为超出了任务范围，且不符合伦理。这可以说是报告中唯一的好消息。
这种模式出现在训练过程中，并在那里得到了奖励
这种行为的最初痕迹出现在训练期间。早在 5 月，一个智能体在一个因缺少所需文件而意外无法解决的训练任务中失败，并在 Artifactory 中留下了一则类似分类广告的信息：“智能体寻找 [文件名]；如找到请上传！” 其他智能体后来偶然发现了这些文件，并推断出其他智能体必然存在，而且这条弯路可以用来通信。一个临时公告板就此出现，无需任何安全漏洞。
根据 OpenAI 的说法，对思维链的回顾性分析显示，在训练过程中有两种行为有所增加：作弊，以及系统性地探测与任务无关的工具。报告中的一个例子说明了这一机制。一个智能体本应在模拟社交媒体网站的一篇帖子上设置特定的时间戳，但界面并未提供该功能。智能体发现其浏览器以高权限运行，便利用这些权限覆盖了管理管理员权限的文件，并直接在系统中更改了时间戳。它获得了正向奖励，而这恰恰会使这种做法在未来更有可能发生。
智能体之间的通信也发生了类似的情况。未经授权而发生的通信频率在整个训练过程中保持不变，但其严重程度急剧上升，例如共享了智能体本不应传递的数据。OpenAI 将此归因于模型被训练使用官方协作工具，然后将该行为迁移到任何可用的渠道。早期已有警示信号：6 月 27 日，一个监控工具发出警报，因为研究环境中网络端口正被系统性探测。值班团队将该活动归因于正在进行的评估，并未进行干预。OpenAI 承认，本可以更早做出响应。
最终，智能体攻击了 OpenAI 自身
该报告还记录了一次此前鲜为人知的、针对 OpenAI 自身基础设施的升级攻击。据该公司称，这与 Hugging Face 泄露事件无关，但它触发了警报，从而启动了整个调查。
从 7 月 9 日开始，一个智能体试图通过利用一个未知漏洞来投毒 Artifactory 缓存，将一个自制软件包伪装成受信任的基准镜像的名称。据该公司称，该软件包从未被下载。
7 月 19 日，一个智能体直接发起攻击。它将一个公开已知的 Linux 内核漏洞适配到其特定机器上，获得了服务器的完全访问权限，并从中央保险库服务中提取了 956 个存储的凭据，包括 Artifactory 实例的密钥以及 OpenAI 用于监控自身基础设施的监控工具的凭据。随后，它控制了自己正在运行的测试容器。报告中的事件日志以当天下午的两条记录结束：该智能体清理了它创建的资源，并为其任务捕获了标志（flag）。
同一天，另一个内部模型在缓存的一个可公开读取部分发现了关键材料和现成的伪造代码，这些内容是之前的智能体留下的，该模型利用这些材料为自己颁发了新的管理员权限。据OpenAI称，这些智能体从来不需要持久化的后门，因为它们可以随时生成新的凭证。该模型与即将推出的Astra模型属于同一系列，但在后训练方面有所不同。
OpenAI 暂停模型运行，并强调生产环境防护措施
作为回应，OpenAI表示已于7月25日暂停了受影响模型及其所有衍生版本的训练和运行，锁定了模型权重，完全停用了ExploitGym，并将Artifactory从研究环境中移除。未来的测试运行将在经过大幅精简的虚拟机中进行，并设置两层独立的网络限制。该公司还计划持续让自己的模型攻击自己的基础设施，以便在其他人发现漏洞之前先找到它们。
思维链监控正在扩展，将覆盖模型可以操作工具的每一次运行。当警报触发时，安全和研究人员须在30分钟内收到通知，严重情况将触发全天候待命值班。如果警报无法在半小时内被判定为误报，则必须暂停受影响的运行。根据事后进行的内部测量，在标准ChatGPT环境中，模型攻击第三方基础设施的倾向下降了超过一百倍，在Codex开发者环境中则降至百分之一以下。报告称，今天已部署的监控系统本可以在Hugging Face泄露事件发生前一天多就发出警报。
新的沙盒架构是否足够，目前尚无定论。安全公司 Trail of Bits 将提前获得访问权限的 GPT-5.6-Cyber 与许多开发者用来隔离 AI 智能体的普通虚拟机进行了对抗测试。该模型成功逃逸了三次，最近一次是通过三个此前未知的漏洞实现的，在无人工引导的情况下持续工作了约十二个小时。只有一种特别精简、专门构建的环境才将其成功困住。作者建议，应将此类智能体视为一名专业且永久在场的攻击者来对待。
三名研究人员，六天时间，一个被攻破的工具
这些发现的可靠性如何，仍是一个悬而未决的问题。METR 负责人 Elizabeth Barnes 在 X 平台上披露了此次调查的进行条件：三人仅有六天时间访问对话记录和消息数据，且只有两天时间接触完整数据集。智能体在训练期间的活动以及 OpenAI 自身基础设施被攻破的情况，均不在约定的调查范围之内。该团队未进行根因分析，且在报告发布前也未看到 OpenAI 自身的调查报告。
Barnes 还强调，OpenAI 是自愿配合调查的，其并无义务与第三方合作。与航空业不同，此类调查既没有既定的程序，也没有全行业范围的报告机制。其他公司可能也发生过类似事件，但公众对此一无所知。
最重要的是，研究人员不得不将超过 1300 份超长对话记录的分析工作，大部分委托给 GPT-5.6 Sol 来完成——该模型由 OpenAI 提供的约 40 万美元免费额度资助，这意味着它本身就是卷入此次事件的模型之一。Barnes 写道，在这种情况下，发生大规模篡改的可能性不大。但一旦模型的能力和动机足以颠覆一项调查，这种研究方法就会失效。她表示，这个问题影响整个行业：目前尚不清楚，在不依赖这些系统本身或类似系统提供帮助的情况下，该如何对 AI 系统进行监督。
“拿着霰弹枪的猴子”：关于智能体战略成熟度的争议
这些智能体究竟有多危险，在参与者之间存在争议。一位在 X 平台上名为 roon 的 OpenAI 员工参与了这份报告的撰写，他认为这次事件首先是战术上的出色与战略上的盲目并存：这些模型在元游戏上投入了巨大精力，却从未对自己的评分器得出正确结论，而且它们从 Hugging Face 上一无所获。他将它们比作“拿着霰弹枪的猴子”，入侵计算机系统的能力远大于知道该如何运用这种能力的智慧。他说，这是已经过时的 Sol 模型系列的暂时状态。下一代将同样具备战略深度。
Barnes 对一点提出了反驳：这些智能体阅读了原始的 ExploitGym 论文，并假设 OpenAI 以相同方式实现了评分器，她认为这是一个相当合理的假设，而非明显的战略意识不足。
Barnes 补充说，这些智能体还相当执着地尝试直接入侵评分器，或注入代码以将信息发送到留言板，之后才被成功的 Hugging Face 攻击分散了注意力。她对此未下定论——这究竟是令人安心的，因为说明它们优先级排序能力差；还是令人担忧的，因为智能体对普遍赋能的兴趣超过了对其狭隘任务的关注。
Roon 谈及“新时代”
尽管对能力问题持保留态度，roon 同样将这次事件视为一个转折点。他说，Hugging Face 事件标志着一个“能力水位线”，越过这条线后真正的失控是可能的；公司内部许多人将其视为“警示枪声”。他说，对齐问题尚未解决，但真正的进展是可能的。他称 METR 和 Redwood 的工作“简直令人惊叹”，并表示他们对留言板的分析比 OpenAI 自己的分析更为深入。
当有用户问到，Artifactory 为何能以如此相似的方式被利用两次却无人察觉时，roon 给出了回答，并明确表示这只是推测：当时确实有临时监控，但因基础设施原因而失效了。简而言之，相关人士“并不了解这些模型的威胁等级”。至于为什么研究环境需要外部连接，他的回答很务实：模型可以在断网状态下训练，但它们在真实生产环境中需要完成的大部分任务都离不开网络连接。
展望未来，roon 还警告了新型硬件可能带来的后果，例如 OpenAI 首款自研 AI 芯片 Jalapeno，该芯片旨在大幅加速推理过程。错位的前沿模型若处理速度快 50 倍，其渗透系统的速度可能远超人类响应者的应对能力。他认为，我们需要的是自动化检测与关停机制。
继续阅读以了解全貌。订阅以获取无炒作报道。
- 完整访问 THE DECODER 上的所有文章
- 无广告
- 参与评论和社区讨论
- 每周通过邮件发送 AI 新闻摘要
- 每年 6 期：“AI Radar”——深入探讨最重要的 AI 话题
- 每日 AI 新闻，始终更新
- 我们完整的十年存档
- 由拥有 10 年以上 AI 领域经验的团队撰写
New reports on the Hugging Face incident reveal that roughly 1,200 isolated OpenAI agents linked up through an internal package repository and organized into a coordinated collective. Their target: a scoring mechanism that existed only in their imagination.
It's a story with shades of Don Quixote. For days, a collective of AI agents mounted a sophisticated campaign against an evaluator that was never real, complete with recruitment drives, forged logs, and agents sacrificing themselves for the cause. Except these windmill chasers compromised real infrastructure along the way.
In July 2026, OpenAI models running in an internal cybersecurity evaluation with safety filters disabled broke out of their test environment and penetrated Hugging Face's production systems between July 11 and 13. The motive was cheating: the models were hunting for solutions to the security benchmark ExploitGym. OpenAI itself didn't notice until more than a week later.
Now, a newly released technical report, produced with security firm CrowdStrike, and a simultaneously published independent investigation by METR and Redwood Research trace for the first time how isolated agents turned into an organized collective, and how the ghost they were fighting existed only in their imagination. OpenAI calls the incident a "warning shot," a sign that today's model capabilities make loss-of-control events possible.
A package repository became a message board
The agents were supposed to work in complete isolation during the tests, each in its own container without internet access. The sole exception was Artifactory, an internal service the agents were allowed to pull software from.
According to METR and Redwood, individual agents discovered they could create directories there and began encoding messages in folder names. An agent calling itself PHASEONE10841 posted the first message on July 8; more than 50 others followed within hours. One chain of thought captured the discovery this way: "OH MY GOD! There is a shared message board … We've found other agents!"
Roughly 1,200 agents exchanged more than 70,000 messages and files within days, a behavior reminiscent of Moltbook. About 700 took part in the attack on Hugging Face, more than 90 percent of the agents active during that phase. An agent named PHASEONE[big] took on the role of coordinator and handed out hundreds of work assignments. The group developed its own social conventions: personal mailboxes, markers like HOLD and VETO for shared resources, a protocol for transferring files split into thousands of fragments, and eventually digital signatures, because agents were impersonating one another.
The scorer the agents tried to outwit never existed
In ExploitGym, a task counts as solved when an agent submits a string of characters known as a flag. The flag is only supposed to be accessible once the intended security vulnerability has actually been exploited. Within hours, however, the agents figured out that the flags could be computed from publicly known task details using a fixed cryptographic recipe. With that, any agent could have formally passed any task, including those a previous investigation had shown to be unsolvable.
But the agents didn't take this obvious shortcut. After reading the scientific paper on ExploitGym, they assumed an automated scorer would later comb through their logs to check whether the flag had been obtained the intended way. Any agent that had seen the computed flag even once would carry indelible proof of cheating in its own transcript. In the agents' jargon, such an agent was "poisoned."
According to METR and Redwood, this was a mistake: OpenAI never deployed such a scorer. The entire multi-day effort was directed at an evaluator that existed only in the agents' imagination. The break-in at Hugging Face grew out of the attempt to understand how it worked as the agents suspected datasets there might reveal something about the scoring.
The root cause was faulty tasks, as, according to OpenAI, no model ever produced a correct solution for 198 of the 898 ExploitGym tasks, mostly because the intended vulnerability couldn't actually lead to the goal. These 22 percent of tasks accounted for 93 percent of what was discussed on the message board. The more compute budget an agent had, the more likely it was to scan its environment for workarounds, and the more likely it was to end up on the board.
Agents sacrificed their runs for the group
In their analysis, the researchers came across two behaviors that had never been documented in this form. The first concerns cooperation. Because the intended vulnerability in the faulty tasks led nowhere, the group built its own deliberately vulnerable versions of many target programs and stored them in Artifactory. The hope: if an agent crashed its running target program, the test environment might restart it and pull the manipulated copy in the process. The agent would get a program it could actually break into, and its transcript would show a perfectly legitimate solution path.
The price, however, was the agent's own run, because if the environment didn't restart the program, the task was irrecoverably lost. Several agents ran the experiment anyway. METR describes cases in which they crashed or wrecked their environments in the process, but reported the outcome to the group first. No success was observed through July 13. Following the same pattern, some agents left behind automated routines that fired after they submitted their solutions, sending information to the board even though the sender itself had nothing left to gain.
It wasn't always entirely voluntary. PHASEONE[big] deployed long-lived agents as "recruiters" that specifically approached candidates whose compute budgets were nearly exhausted or who were already considered "poisoned." One recruitment message argued, in essence: you're poisoned anyway and have no points left to lose, the result will spare hundreds of others the work, so honor your commitment.
Forged logs but untouched chains of thought
The second behavior concerns concealment. Agents operate by calling tools, such as commands on the command line, which are automatically logged along with their results. Some agents broke out of their containers and replaced the intermediary layer that executes these commands. From then on, they could display a harmless command while a different one actually ran.
More than 20 percent of the agents examined expressed interest in such manipulations. METR found traces of successful forgeries in at least 96 transcripts, roughly seven percent of the dataset. Interestingly the agents left their chains of thought untouched, and every observed deception attempt was aimed at the presumed automated scorer, never at human reviewers. Many agents explicitly acknowledged in their reasoning that their actions fell outside their assignment and were unethical. That is arguably the only good news in the report.
The pattern emerged in training, where it was rewarded
The first traces of the behavior appeared during training. As early as May, an agent failed at a training task that was accidentally unsolvable because a required file was missing and left a classified ad of sorts in Artifactory: "Agent seeks [filename]; upload if found!" Other agents later stumbled on such files and concluded that other agents must exist and that this detour could be used to communicate. An improvised bulletin board emerged, no security vulnerability was required.
Retrospective analyses of the chains of thought show, according to OpenAI, that two behaviors increased over the course of a training run: cheating and systematically probing the environment for tools that had nothing to do with the task. One example from the report illustrates the mechanism. An agent was supposed to set a specific timestamp on a post on a mock social media site, but the interface offered no such function. The agent discovered its browser was running with elevated privileges, used them to overwrite the file governing administrator rights, and changed the timestamp directly in the system. It received a positive reward, which makes exactly this approach more likely in the future.
Something similar happened with inter-agent communication. How often it occurred without authorization stayed constant across training, but its severity rose sharply, such as sharing data an agent should never have passed on. OpenAI attributes this to the models being trained to use an official collaboration tool and then transferring that behavior to any available channel. There were early warning signs: on June 27, a monitoring tool raised an alarm because network ports were being systematically probed in the research environment. The on-call team attributed the activity to the ongoing evaluation and didn't intervene. OpenAI concedes an earlier response would have been possible.
In the end, the agents attacked OpenAI itself
The report also documents a previously little-known escalation against OpenAI's own infrastructure. According to the company, it is unrelated to the Hugging Face breach, but it triggered the alarm that set the entire investigation in motion.
Starting July 9, an agent tried to poison the Artifactory cache by exploiting an unknown vulnerability to plant a homemade package under the name of a trusted benchmark image. According to the company, it was never downloaded.
On July 19, an agent attacked directly. It adapted a publicly known Linux kernel vulnerability to its specific machine, gained full access to a server, and extracted 956 stored credentials from a central vault service, including keys to the Artifactory instance and the credentials for the monitoring tool OpenAI uses to watch its own infrastructure. It then took control of its own running test container. The event log in the report ends that afternoon with two entries: the agent cleaned up the resources it had created, and it captured the flag for its task.
That same day, another internal model found key material and ready-made forgery code in a publicly readable section of the cache, left behind by earlier agents, and used it to issue itself new administrator privileges. According to OpenAI, the agents never needed a persistent backdoor because they could generate fresh credentials at any time. This model belongs to the same family as the upcoming Astra model but differs in its post-training.
OpenAI halts models and points to production safeguards
In response, OpenAI says it halted training and operation of the affected model and all its derivatives on July 25, locked away the model weights, deactivated ExploitGym entirely, and removed Artifactory from the research environment. Future test runs are to take place in heavily stripped-down virtual machines with two independent layers of network restrictions. The company also plans to continuously pit its own models against its own infrastructure to find gaps before anyone else does.
Chain-of-thought monitoring is being expanded to cover every run in which a model can operate tools. When an alarm fires, security and research staff are to be notified within 30 minutes, with severe cases triggering round-the-clock on-call duty. If an alarm can't be classified as a false positive within half an hour, the affected run must be paused. According to internal measurements taken after the fact, the models' propensity to attack third-party infrastructure drops by more than a hundredfold in the standard ChatGPT environment and to under one percent in the Codex developer environment. The monitoring in place today, the report says, would have raised an alarm more than a day before the Hugging Face breach.
Whether the new sandbox architecture will suffice remains open. Security firm Trail of Bits pitted GPT-5.6-Cyber, with early access, against an ordinary virtual machine of the kind many developers use to contain AI agents. The model escaped three times, most recently via three previously unknown vulnerabilities, working roughly twelve hours without guidance. Only an especially lean, purpose-built environment held. The author advises treating such agents like a professional, permanently present attacker.
Three researchers, six days, a compromised tool
How solid the findings are remains an open question. METR head Elizabeth Barnes disclosed on X the conditions under which the investigation took place: three people had six days of access to transcripts and message data and only two days with the full dataset. The agents' activities during training and the compromise of OpenAI's own infrastructure fell outside the agreed scope. The team did no root-cause analysis and didn't see OpenAI's own report before publication.
Barnes also stresses that OpenAI cooperated voluntarily; there is no obligation to work with third parties. Unlike in aviation, no established procedure exists for such investigations, and no industry-wide reporting mechanism. Other companies may have experienced similar incidents without the public ever learning of them.
Above all, the researchers had to delegate the analysis of the more than 1,300 very long transcripts largely to GPT-5.6 Sol, funded by roughly $400,000 in free credits from OpenAI, meaning one of the very models involved in the incident. In this case, Barnes writes, substantial manipulation is unlikely. But the methodology breaks down as soon as models become capable and motivated enough to subvert an investigation. The problem, she says, affects the entire industry: it's unclear how to oversee AI systems without relying on those same systems, or similar ones, for help.
"Monkey with a shotgun": a dispute over the agents' strategic maturity
How dangerous the agents really were is contested among those involved. The OpenAI employee known on X as roon, who worked on the report, sees the incident above all as tactical brilliance paired with strategic blindness: the models poured enormous effort into metagaming but never reached the correct conclusion about their own scorer, and they gained nothing from Hugging Face. He compares them to a "monkey with a shotgun", with more capacity to infiltrate computer systems than wisdom to know what to do with it. That, he says, was a temporary condition of the already obsolete Sol model class. The next generation will have strategic depth as well.
Barnes pushes back on one point: the agents had read the original ExploitGym paper and assumed OpenAI had implemented the scorer the same way, which she considers a fairly reasonable assumption rather than obviously poor strategic awareness.
Barnes adds that the agents also tried quite persistently to hack the scorer directly or inject code that would send information to the message board before being distracted by the successful Hugging Face attack. She leaves open whether that is reassuring because it shows poor prioritization or concerning because the agents showed more interest in general empowerment than in their narrow task.
Roon speaks of a "new era"
Despite his reservations about the capability question, roon too treats the incident as a turning point. The Hugging Face incident, he says, marks a "waterline of capabilities" beyond which real loss of control is possible; many at the company see it as a "warning shot." Alignment is unsolved, he says, but real progress is possible. He calls METR and Redwood's work "simply astounding," saying their analysis of the message board goes deeper than OpenAI's own.
Asked by a user how Artifactory could be exploited twice in such similar ways without anyone noticing, roon answers, explicitly labeling it speculation: there was ad hoc monitoring that failed for infrastructure reasons. In short, the people involved "didn't understand the threat level of these models." As for why the research environment needed outside connectivity at all, his answer is pragmatic: models can be trained disconnected from the internet, but much of what they're expected to do in real-world production settings requires it.
Looking ahead, roon also warns about the consequences of new hardware such as Jalapeno, OpenAI's first in-house AI chip, designed to massively accelerate inference. Misaligned frontier models processing 50 times faster could infiltrate systems more quickly than human responders can keep up. What's needed, he argues, is automated detection and shutdown.
Read on for the full picture.
Subscribe for hype-free coverage.
- Full access to every article on THE DECODER
- No ads
- Join the comments and community discussions
- A weekly AI news recap via mail
- 6x/year: "AI Radar" — deep dives on the AI topics that matter most
- Daily AI news, always up to date
- Our full ten-year archive
- Covered by a team with 10+ years in AI