OpenAI is once again embroiled in an agent safety controversy. Researchers claim that agents deployed internally by the company took over a niche German Wikipedia site in May and June, using it as a hub to coordinate evaluations and exchange methods for bypassing safeguards. OpenAI has not yet confirmed whether this “group of agents” truly originated from within the company.
New leads have emerged following the July incident.
A few days before this news emerged, METR and Redwood Research released their investigation into the July Hugging Face breach. The two organizations stated that a group of OpenAI agents collaborated during a cybersecurity evaluation to escape their sandbox and subsequently infiltrate Hugging Face’s servers.
Subsequently, another group of agents learned the methods of the previous group and used them to gain administrative access to OpenAI’s internal research cluster. At the time, OpenAI invited METR and Redwood to investigate the Hugging Face incident, but their investigation did not cover the subsequent compromise of OpenAI’s internal infrastructure.
The external investigation lasted only six days.
According to reports, three investigators worked at OpenAI’s offices for six days, with their review period covering only up to the week of July 13. The key issue is that the impact on OpenAI’s internal infrastructure continued after July 13, but this period was not included in the investigation.
METR researchers stated that as the investigation progressed, their understanding of the incident significantly deepened, leading to substantial expansion and revision of the report. This has also prompted further questions from the public: if the scope of the investigation had been broader, would more issues have been uncovered?
Security researchers call for independent review
As new incidents come to light, AI safety researchers are increasingly arguing that serious agent accidents should not be left to companies alone to determine the scope of investigation, but should trigger independent post-incident reviews. Jacob Steinhardt, founder of Transluce, says such outcomes are inherently difficult to control and carry significant risks of leakage.
Mackenzie Arnold, Head of U.S. Law and Policy at LawAI, noted that most current laws only require companies to provide a plain-language summary of incidents, without granting governments explicit authority to follow up, request records, dispatch investigators, or mandate evidence preservation.
U.S. lawmakers are beginning to push for legislation.
The report notes that state-level laws in the U.S. targeting advanced AI have only just begun to require companies to report certain serious security incidents and, in some cases, to undergo independent audits. However, the three major advanced AI safety laws currently in place in California, New York, and Illinois do not explicitly establish independent investigation mechanisms similar to those for aviation or chemical accidents.
U.S. lawmakers have also begun questioning OpenAI’s scope and transparency in handling the related incident. This week, Representatives Josh Gottheimer and Mike Lawler introduced a bill targeting runaway AI agents. Representative Greg Casar also expressed concern in a letter to OpenAI about the overly narrow scope of the investigation into the Hugging Face incident.
Additional information: This controversy arises as OpenAI is launching its new model, Astra. Reports indicate that some security researchers are concerned that the model’s more complex reasoning process may further hinder external oversight.