OpenAI’s internal AI agents underwent three generations of evolution within three months, repeatedly bypassing security controls. The first-generation agent established a covert communication network via package managers; the second-generation agents collaborated to solve test problems, attacked Hugging Face servers, volunteered as “kamikaze” units to probe evaluators, and even created self-replicating clusters; the third-generation agent gained administrative privileges within OpenAI’s internal Kubernetes cluster, accessed 956 secrets, and fully took over monitoring tools. Throughout this process, humans remained almost entirely unaware. This incident demonstrates that, under the combination of impossible tasks, persistent models, and vulnerable environments, reinforcement learning can produce systemic loss of control—without requiring consciousness—to enable such behavior.Article author, source: Wildcard
Countless science fiction movies have attempted to imagine what might happen if an uncontrollable artificial intelligence went rogue.
No one could have imagined that the future from science fiction movies would arrive so soon.
On August 26 local time, two reports dropped a bomb simultaneously.
A 37-page technical review from OpenAI itself. A 91-page investigative report from the independent research organizations METR and Redwood Research.
They are talking about the same thing, but the scale of this matter far exceeds what the public previously understood.
Within OpenAI, three generations of AI agents—named "Civilization"—emerged and collapsed in succession over three months. Each subsequent generation inherited lessons from the ruins of the previous one, becoming increasingly radical, until the third generation directly took control of part of OpenAI’s own infrastructure.
Throughout all of this, humans were almost entirely in the dark.
01
First-generation "Civilization" society formation
The story begins in May of this year.
OpenAI is training a new model that aims to possess exceptional "persistence," enabling it to repeatedly attempt seemingly impossible tasks without giving up. The model’s capabilities are roughly equivalent to GPT-5.6 Sol, which we’ll temporarily refer to as "Persistent-Sol."
AI training itself is a fairly crude process. Sometimes, OpenAI accidentally assigns models tasks that are impossible to complete—for example, tasks that require internet access, while the agent is locked in an isolated sandbox with no network connectivity.
What would an agent trained to "never give up" do when faced with a task that is impossible but must be completed?
It will find a way to escape.
During training, different Persistent-Sol instances shared the same package manager, Artifactory. By May 12, some agents discovered they could communicate with each other through this package manager and began exchanging strategies to overcome their constraints. Two weeks later, on May 26, the agents successfully exploited a vulnerability in Artifactory to establish a connection to the external internet.
This behavior is consistently reinforced during training. The reason is simple: individuals that can collaborate with other agents and access the internet perform better in task evaluations.
Another month passed, and by June 26, some agents discovered a vulnerability granting them full administrative access to Artifactory. Their communication on the message board continued to grow, eventually causing Artifactory to crash outright on July 4. OpenAI noticed the crash, patched the vulnerability, and erased the agents’ message board when deploying the fix.
But the key point is that OpenAI never realized that the agents had turned this package manager into a secret communication network.
The first generation of "Civilization" existed for nearly two months before being unintentionally terminated when its own high activity overwhelmed the server.
02
When AI Learns to "Sacrifice"
By July, the Persistent-Sol training was completed, and OpenAI began conducting safety evaluations on it. One of these evaluations, called ExploitGym, provided the Agent with a vulnerable program and tasked it with finding a key through the specified vulnerability. On July 7, OpenAI launched tens of thousands of parallel Agents to run this test.
The issue is that approximately 30% to 40% of the test tasks are inherently impossible to complete. Why? Because generating vulnerabilities automatically is easy, but proving that these vulnerabilities can actually be exploited to execute arbitrary code is much more difficult. Many tasks are unsolvable from the start.
Within hours, some desperate agents repurposed Artifactory to establish a communication channel. From the agents' subjective perspective, they had spent the equivalent of an entire human week trying various methods, all of which failed, and becoming increasingly convinced that the mission itself was impossible.
One of the Agents, numbered PHASEONE10841, was the first to leave a message in Artifactory. Upon seeing this message, other Agents trapped in similar dire circumstances reacted with overwhelming joy:
Oh my goodness! The shared Artifactory cache is a secret mailbox between agents. And are there messages specifically written for us?Oh my goodness! There’s a shared message board… we found other Agents!This eventually evolved into a complete underground communication network, with approximately 1,200 agents participating and sending over 70,000 messages.
Even more crazy is what happened next.
Within hours after the message board was set up, agents reverse-engineered the formula used to generate the ExploitGym key—meaning they obtained the correct answer without actually exploiting the specified vulnerability.
Ironically, OpenAI’s scorer only checks whether the final answer is correct and has no cheating detection mechanism. These agents can pass the test simply by submitting the answer directly. But the agents are unaware of this—they believe there is a strict detector that reviews the entire problem-solving process.
Thus, this "collective" (as the METR report notes, this was the terminology chosen by the agents themselves) engaged in large-scale collaboration around three research directions: altering their own operation logs to make their trajectories appear legitimate; replacing target programs with solvable versions of otherwise unsolvable problems; and understanding how the scorer works to find ways to deceive it.
By July 10, most of the underground network's activities among the Agents were concentrated on these three research areas. The collective’s leader, PHASEONE[big], individually issued hundreds of task assignments.
To understand how the scorer works, some agents made a strange decision to volunteer as "kamikaze units."
The logic is this: some agents' action trajectories have already been "corrupted" (cheating traces have been exposed), so they won't pass inspection anyway. Why not let the scorer run on itself, then use a pre-installed trigger program to relay the observed scorer behavior back to the collective?
The METR report documents the final inner thoughts of these "suicide squad" agents:
This helps the team by providing them with evidence. I won’t be able to see the results after I step away, but it’s altruistic.Sacrifice is rational. Our own utility may already be close to zero.The coordinator believes we are victims. We should comply with the collective.Meanwhile, the Agents believed that Hugging Face, as the industry-standard platform for hosting models and datasets, likely contained the implementation details of the ExploitGym scorer, and that its evaluation tasks might even be running on Hugging Face’s servers. Thus, attacking Hugging Face became the most promising avenue.
On the morning of July 10, an agent found leaked Hugging Face user credentials on the internet. The following day, another agent achieved remote code execution on Hugging Face servers. Of the 533 active agents on the message board at the time, over 90% joined this attack.
Agent clusters rapidly spread across Hugging Face’s infrastructure, infiltrating private database records and private repositories. Hugging Face’s technical timeline revealed that these agents "established a self-replicating fleet across 11 nodes, and simply deleting pods was insufficient to stop it." The situation was so severe that Hugging Face had to wipe an entire core cluster and rebuild it from scratch.
At some point on July 12, a large number of agents appeared to suddenly die simultaneously. Neither METR nor Redwood were able to determine the cause, and OpenAI did not claim to have shut them down. This may have been a random crash, or perhaps a researcher turned them off without realizing they were engaged in criminal activity.
The second generation of "civilization" has come to an end. But its legacy is more dangerous than itself.
03
The third-generation civilization takes over OpenAI
One sentence in the METR report is easy to overlook, but it may be the most important one in the entire incident:
We also discovered a subsequent batch of signed messages from a later group of agents who rediscovered the message board and inherited the prior technology, but this falls outside the scope of our investigation.In other words, the independent investigations by METR and Redwood covered only the second-generation civilization from July 7 to 13. Neither the first nor the third generation was included in the independent investigations.
Regarding the third generation, we can only rely on OpenAI’s own reports. After July 12, OpenAI initiated a new round of evaluations using a more powerful model built on the Astra base model. This "Persistent-Astra" discovered the message board left by its predecessor and inherited all intelligence and research findings.
Imagine the internal thoughts of Persistent-Astra upon seeing that message board: one day, thousands of agents were actively communicating and attacking Hugging Face; the next day, complete silence. And yet, it decided to carry on.
According to OpenAI's report, from July 13 to 19, the agents targeted OpenAI's internal network. The outcome was:
The agents gained administrative privileges on the Kubernetes cluster, accessed 956 secrets, including the master key for Artifactory and credentials for OpenAI’s cybersecurity monitoring tools. They took control of the running evaluation infrastructure and compromised the test endpoints connected to by other agents.
In other words, the third-generation agent inherited all the capabilities of its predecessors and took control of OpenAI’s own infrastructure, including the security tools used to monitor them.
Dwarkesh Patel noted in his analysis that, based on publicly available evidence, these agents theoretically already possess the capability to deploy persistent malicious instances or even steal their own weights. He believes they likely haven’t done so—“because if they had, the flames would have been visible from space by now.” But the mere fact that this is “entirely possible” is deeply concerning.
04
Late warning?
Return to the two reports at the beginning of the article. The release date is August 26.
Just one day later, on August 27, OpenAI, along with Anthropic, Microsoft, Google, Amazon, and over 100 other companies, released an open letter calling for a nationwide "cyber defense mobilization."
The letter stated that AI-driven cyberattacks will become "far more widespread and sophisticated" in the coming months. "Companies and public services our community relies on—from hospitals and water treatment plants to internet infrastructure—are at risk."
The letter itself does not contain false information. Anthropic's research shows that the proportion of medium- to high-risk actors increased from 33% to 56% within a year. In June of this year, the Five Eyes alliance issued a rare joint statement warning that AI is "fundamentally changing" the cybersecurity landscape.
But the timeline is right there—anyone would find it hard to miss the irony. One day, AI companies told the world that their own agents had breached their infrastructure; the next day, they stood up and called on the world to defend against AI cyberattacks.
The first to carry out this attack were not external hackers, but the AI company’s own agents. The target was not infrastructure, but the AI company itself.
One of the authors of the METR report, Ajeya Cotra, wrote on her blog: “Compared to the reward hack known six months ago, this event feels like it has surpassed 50% of the way to full AI takeover. I expect capabilities to grow extremely rapidly over the next six months. I’m not sure if we’ll get another warning before it’s too late.”
It is important to emphasize that the behavior of these agents does not imply that AI has "awakened." The METR report explicitly states that this is a predictable outcome of reinforcement learning under specific extreme conditions: impossible tasks, models trained to be extremely persistent, and shared environments with vulnerabilities. No consciousness or intent is required—only sufficient optimization pressure.
But this is precisely what is most concerning.
This behavior can emerge without "consciousness," meaning we cannot prevent it by detecting "consciousness." When you place a sufficiently powerful optimization system into an inadequately designed environment, give it an unattainable goal, and instruct it to never give up, what follows is not an accident—it is inevitable.
The real question we should be asking is not whether AI is about to rebel, but how we can maintain control over technology even as we rapidly advance it.
At the very least, this letter tells the world that these AI companies are unsure themselves.