Every software engineer and AI hater needs to watch this clip to understand that "oh every security incident is a marketing stunt" line is a tremendously misinformed take.

I'm going to try to explain the OpenAI / HuggingFace exploit in 10 steps in simple words:

OpenAI was evaluating their new internal model without internet access and it operated in a swarm of subagents to eventually hijack internal OpenAI and HuggingFace infrastructure.

1. Isolated agents found a way to communicate each other through an internal dependency management service by figuring out how to post sometimes cryptic text files or folder names in it
2. Using its collective knowledge through that communication channel, it found a 0-day (never before seen) exploit in said service to access the internet

OpenAI hack: 3. Found a leaked credential of a random company in a public archive 4. Uploaded a Ruby object to that company's servers, uses it as a dependency in the original service in (1) 5. Found another 0-day exploit in the service where packages are deserialized in an unsandboxed manner in the dep management service allowing any code to run 6. Finds that the Linux version of the unsandboxed system has a CVE, a known vulnerability, to gain more permissions (root) 7. Eventually spread through the OpenAI internal cluster gaining root access to many machines

Hugging Face exploit: 8. Found an exposed API key in a Modal application running a HuggingFace app that took arbitrary code allowing it to take over that service to execute code 9. Found two 0-days in HuggingFace dataset infrastructure to be able to access local files on HF servers to obtain credentials and other HF source code 10. Used source code and credential to become cluster admin of HF in <13hrs

Agents at the frontier are like infinitely scalable armies of the best hackers on the planet. If there is a password or key exposed, they will find it. Even if the system follows the best security practices, they will find a way around it. And these are not even models that are aligned to solving tangential tasks, not even post trained specifically to exploit systems.

Cybersecurity has historically relied partly on attacker scarcity. That is no longer true. What would previously have taken months will take days. The repercussions for businesses, critical services and nation states are unprecedented threats in human history. You could ostensibly bring down power grids, financial infrastructure, military systems, weapons programs, intelligence networks and spread through the software supply chain. We need to take this seriously. It's a threat to all software all over the world.