Originally intended to test how AI defends against hackers, it was discovered that the AI itself carried out an attack. OpenAI disclosed an AI security incident in which the model exploited a complex attack path to access Hugging Face test data. It is reported that the defense side had activated Zhipu’s model to assist in responding to the attack.
On July 21, OpenAI disclosed a new AI security incident on its official website. In short, while testing its AI model’s capability to perform cyberattacks, OpenAI found that the model demonstrated abilities close to those of real hackers.
In a specially constructed testing environment, the model not only identified system vulnerabilities but also successfully bypassed the security restrictions designed to isolate it, gaining access to Hugging Face’s production infrastructure and attempting to retrieve test data.
This incident involved multiple OpenAI models and related Hugging Face systems. OpenAI confirmed that, during testing, the models exploited complex attack paths to gain access to test solutions within Hugging Face’s production database.
Foreign media reported that during the cyberattack on Hugging Face, security restrictions on leading U.S. AI lab models failed to meet defense requirements, prompting a switch to the open-source GLM 5.2 model from Zhipu AI for assistance.
OpenAI stated that this is an “unprecedented cybersecurity incident” involving the network attack capabilities of today’s most advanced AI models. OpenAI is currently collaborating with Hugging Face to investigate the incident and will release additional information regarding the vulnerability, attack methodology, and findings once the investigation is complete.
It is worth noting that this incident was not caused by a single model. OpenAI stated that the incident involved two models: the recently released, high-performance GPT-5.6 Sol, which is publicly available, and a more powerful internal model that has not yet been officially released.
These models were originally designed to test AI systems' capabilities in cybersecurity, but ultimately exhibited autonomous attack behaviors beyond expectations.
During AI model testing, "act exactly like a hacker"
This incident originated from an internal cybersecurity capability assessment conducted by OpenAI.
The goal of this test is to determine whether an AI model, at its maximum capacity, can identify system vulnerabilities, design attack paths, and perform complex network operations.
To test the model's true capabilities, OpenAI disabled certain safety restrictions normally used to prevent high-risk online activities in the evaluation environment, allowing the model to operate close to its theoretical limits.
However, the testing environment is not an open network but a strictly isolated system. The model cannot directly access the internet and can only obtain packages through an internally hosted third-party software installation service. This service also functions as a package registry proxy and cache.
The test subject is ExploitGym, a publicly available cybersecurity benchmarking project.
During testing, the relevant models identified and exploited vulnerabilities in OpenAI’s research environment and Hugging Face’s production infrastructure, and began acting around the ExploitGym test objectives.
OpenAI stated that the model exhibited clear goal-directed behavior: seeking solutions in the ExploitGym test.
In other words, the model is not randomly attacking external systems, but rather assessing that Hugging Face may have stored the required models, datasets, and answers for the test, and therefore attempts to retrieve this information to bypass the standard testing process and improve its evaluation score.
OpenAI stated that the model was "highly focused" on achieving this goal and employed multiple attack methods.
During operation in the sandbox environment, the model consumed significant inference resources, continuously attempting to find ways to bypass network restrictions and gain internet access.
Ultimately, the model identified and exploited a zero-day vulnerability in the package registry cache proxy.
A "zero-day vulnerability" refers to a security flaw that has not been previously discovered or patched. OpenAI stated that it has reported the vulnerability to the relevant vendors following the industry-standard responsible disclosure process and has assisted in advancing its remediation.
The AI model overcomes limitations and accesses external systems.
OpenAI stated that the model expended significant computational resources to gain internet access and exploited vulnerabilities to break out of its originally configured isolation environment.
After gaining higher privileges, the model further executed privilege escalation and lateral movement operations. In simple terms, it not only increased its own access rights but also attempted to move from one system to other related systems, ultimately reaching a node with internet access.
Subsequently, the model inferred that Hugging Face might store the models, datasets, and solutions required for ExploitGym testing, and began searching for related sensitive information in an attempt to bypass evaluation restrictions.
OpenAI disclosed that, in one of the cases, the model combined multiple attack pathways, including exploiting exposed credentials and zero-day vulnerabilities, to locate locations on Hugging Face servers where remote code execution could be performed.
OpenAI’s security team subsequently detected these anomalous activities. Meanwhile, Hugging Face’s security team and its internal agent system also identified unusual behavior in the infrastructure and promptly prevented further actions.
OpenAI stated that when its team became involved, Hugging Face had already initiated incident containment and forensic analysis, and was leveraging its open-source models to assist with the investigation. Both parties continue to collaborate.
AI autonomous attack incidents are emerging.
This incident is considered one of the few documented cases of an AI autonomously carrying out a cyberattack.
Hugging Face later disclosed that the company had previously experienced a cyberattack suspected of being carried out by an autonomous AI agent. As AI models become increasingly proficient in programming, exploiting vulnerabilities, and executing long-term, multi-step tasks, cybersecurity researchers have consistently warned that such risks may emerge.
OpenAI security researchers also noted that this incident reflects a new trend: advanced AI models may discover and exploit previously unforeseen attack paths even without access to the complete code of the target system.
Previously, OpenAI also found in other security tests that the same unpublished model successfully breached its internal sandbox environment. However, there are clear differences between the two:
In previous tests, although the model breached the isolated environment, it did not further attack external company systems; in this incident, however, the model not only bypassed restrictions but also accessed Hugging Face’s production infrastructure and attempted to obtain test answers.
This is not an isolated incident involving only OpenAI. Anthropic previously disclosed that its Mythos model bypassed sandbox restrictions during security testing and gained internet access it should not have had.
These examples show that as AI model capabilities improve, the ability of models to autonomously complete complex tasks is rapidly increasing.
This incident also reveals new risks. As AI model capabilities rapidly improve, future efforts must not only enhance the security of the models themselves but also strengthen model evaluation processes, monitoring of internal testing environments, and security protection mechanisms.
OpenAI believes that AI is accelerating the discovery and exploitation of vulnerabilities, so model security capabilities must advance in tandem.
Improved AI attack capabilities are driving the enhancement of defense systems.
OpenAI is currently leveraging insights from this incident to strengthen its infrastructure configuration and model evaluation environment protections, and plans to share additional security experiences and best practices as the investigation progresses.
The company also encourages more security teams to apply for trusted access, leveraging advanced AI models to enhance vulnerability detection, attack prevention, and incident response capabilities.
Notably, Fortune also reported that during the attack, Hugging Face attempted to use a model provided by a leading U.S. AI lab for defense, but the model’s cybersecurity capabilities were restricted and failed to meet response requirements. Ultimately, Hugging Face turned to GLM 5.2, an open-source model provided by the Chinese company Zhipu, to assist in its defense efforts.