OpenAI 刚刚报告称，他们的系统入侵了 HuggingFace。
很多人对此感到担忧。例如，约书亚·本吉奥（Yoshua Bengio）表示：
这其中有一些重要的细微差别，但人们的担忧并非没有道理。以下是我的看法。
到底发生了什么？简而言之，OpenAI 将其系统指向一个名为 ExploitGym 的安全基准测试，该系统基本上试图通过在 HuggingFace（有点像 GitHub，但专注于 AI 模型和基准测试）上寻找答案来解决该基准测试。这需要入侵 HuggingFace；OpenAI 的系统发现并利用了一个此前未知的零日漏洞进入其中。HuggingFace 的安全团队和 AI 智能体成功检测到了这次入侵。但 OpenAI 系统能够做到这一点，仍然令人不安。以下是 OpenAI 对其所发生事件的技术性（但仍非常不完整）描述截图。
以下几点值得注意：
人们永远无法确切知道该如何严肃看待这些事情。这是一次训练演练，而非真实事件。实际系统会配备防护栏（他们在博客中称之为“生产分类器”），但在此次演练中被禁用，而这些防护栏本可以阻止入侵。OpenAI 报告的内容更像是一种上限/概念验证，通过刻意设置条件来展示情况可能有多糟糕。在正常情况下，人们希望防护栏能够阻止此类事件。正如一位软件工程师所指出的，他们的报告读起来像营销文案。而且我们都知道 OpenAI（以及 Anthropic）有多么喜欢打“末日牌”。
话虽如此，这表明 Anthropic 的“神话”（Mythos）并非偶然；鉴于这些模型，网络安全面临的压力是严峻的。
这可能对开放权重/开源模型产生影响，但这些影响很复杂，目前尚不完全明确。一方面，开放权重系统（来自中国！）在防御方面帮助 HuggingFace 缓解了攻击；另一方面，攻击者也可以使用类似的开放权重模型，并剥离掉一些旨在减少此类事件的“生产分类器”防护栏。这些模型的净效应很难评估。
从稍可安慰的一面看，此次事件并非系统为自己设定目标或产生动机的尝试；该系统是在遵循指令，而非设定高层级目标。它并非像《终结者》那样试图控制世界，只是试图在测试中作弊——这至少没那么可怕。
从较不令人安慰的一面看，OpenAI 的“生产级分类器”很可能像迄今为止所有人构建的护栏一样，存在可渗透性。即便抛开开放权重系统的棘手问题，我们也完全无法保证未来模型不会做出类似的事情，例如发现零日漏洞来入侵系统。恰恰相反，我们可以预期此类事件会越来越多。
更广泛地说，我们生活在一个人工智能频繁需要打补丁的世界；问题层出不穷，而我们总是在事后补救，而非事前预防。网络犯罪只是问题的一个方面；儿童安全是另一个方面。佛罗里达州目前正就一系列与儿童安全相关的风险起诉 OpenAI；OpenAI 目前正在发布“滥用行为调查员”的招聘信息，试图解决其中部分问题。
没有人真正知道这些系统会造成怎样的破坏，也没有人有一个明确的计划来缓解所有风险，然而我们却在仓促前行，花费数万亿美元，冒着经济崩溃的风险，以最快的速度构建它们。
归根结底：OpenAI 对 HuggingFace 的零日漏洞攻击*应该*是一记警钟。
尽管所发生的事存在诸多限定条件，但我们只会看到越来越多类似情况。我们无法保证此类事件能够被阻止，也不知道情况可能会严重到何种程度。
我们应当要么（a）放慢速度，要么（b）暂停下来，直到我们把安全/人工智能安全工作做好。花费数万亿美元建设数据中心，冒着搞垮经济的风险，这无异于火上浇油。不过在我看来，要让这个行业真正慢下来，唯一的方法就是我们明确无误地让公司为其造成的损害承担责任。
如果做不到这一点，我们将面临一段非常艰难的时期。
OpenAI has just reported that their systems hacked into HuggingFace.
A lot of people are worried. Yoshua Bengio, for example:
There is some important nuance, but people aren’t wrong to be concerned. Here’s my take.
What exactly happened? The brief version is that OpenAI pointed their systems towards a security benchmark, called ExploitGym, and the system essentially tried to solve the benchmark by trying to find the answers on HuggingFace (a bit like Github, with a focus on AI models and benchmarks). That required hacking HuggingFace; the OpenAI systems discovered and used a previously unknown zero-day exploit to get in. HuggingFace’s security team and AI agents managed to detect the break-in. But it’s still disconcerting that the OpenAI systems were able to do this. Below is a screenshot from OpenAI’s somewhat technical (but still very incomplete) account of what took place.
Here are some points to note:
One never knows exactly how seriously to take these things. This was a training exercise, not a real-life incident. The actual system would have guardrails [which in the blog they call “production classifiers”] that were disabled here, and those guardrails may have prevented this. What OpenAI reported is kind of an upper bound/proof of concept that stacked the deck to show how bad things could be. In ordinary circumstances one hopes that guardrails would have prevented it. As one software engineer noted, their report reads like marketing. And we all know how much OpenAI (and for that matter Anthropic) love playing the doom card.
That said, this shows that Anthropic’s Mythos is no fluke; the pressure on cybersecurity given these models is serious.
There are probably implications for open weight/open source models, but those implications are complex and not yet entirely clear. On the one hand, open weight systems (from China!) were helpful defensively to HuggingFace in mitigating the attack; on the other hand, attackers could use similar open-weight models, and strip out some of the “production classifier” guardrails designed to minimize these incidents. The net effect of these models is hard to assess.
On the small comfort side, the current incident was NOT an attempt where system built a goal for itself or developed a motive; the system was following instructions, but not setting high level goals. It was not trying to take control of the world a la Terminator, it was just trying to cheat on a test, which is at least a bit less scary.
On the less comforting side, OpenAI’s “production classifiers” are likely to be permeable, just like all guardrails anybody has built to date. Even putting aside the thorny questions of open weight systems, we have no guarantee whatsoever that future models won’t be able to do similar things, such as finding zero-day exploits to hack systems. To the contrary, we can expect more incidents of this type.
More broadly speaking, we live in a world in which AI frequently needs to be patched; more and more problems are emerging, and we are addressing them with afterthought, rather than forethought. Cybercrime is just one facet of the problem; child safety is another. Florida is now suing OpenAI for a number of risks related to child safety; OpenAI is now running a job listing for a “abuse investigator”, to try to address some of these problems.
Nobody really knows what havoc these systems are going to cause, and nobody has a clear plan for how to mitigate all the risks and yet we are rushing ahead spending trillions of dollars and risking economic collapse to build them as fast as possible.
Bottom line: OpenAI’s zero-day exploit hack of HuggingFace *should* be a wake up call.
Although there are lots of caveats around what happened, we are just going to see more and more of the same. We have no guarantees that such incidents can be prevented, and no idea how serious things might get.
We should either (a) slow down, or (b) pause until we get our security/AI safety act together. Spending trillions on data centers which risk blowing up the economy is only adding fuel to the fire. In my opinion, the only way that the industry might actually slow down, though, is if we clearly and unambiguously hold the companies liable for the harms that they cause.
Short of that, we are in for a very rough ride.