OpenAI has released a fully automated red teaming system called GPT-Red to identify security vulnerabilities before model deployment. The company states that this tool has been used in the training process of GPT-5.6, with a focus on strengthening defenses against prompt injection attacks.
Use AI to measure AI
Red team testing, originally a common practice in cybersecurity, involves actively attacking systems to identify exploitable vulnerabilities in advance. OpenAI has further automated this process, enabling the model to generate attack samples itself and then use successful cases to train defensive models in reverse.
OpenAI states that GPT-Red, through adversarial self-play training, continuously generates more potent prompt injection attacks. Whenever an attack succeeds, these samples are incorporated into subsequent training to enhance the defense model’s resistance.
Disclosure of Internal Testing Data
According to OpenAI, in internal evaluations, GPT-Red successfully identified exploitable issues in 84% of test scenarios, compared to a 13% success rate for human red teams in similar tests. The company stated that these attack samples were subsequently used to train GPT-5.6, reducing the model’s failure rate on challenging prompt injection benchmarks.
The article also mentions a case: before the vulnerability was patched, GPT-Red manipulated a fleet of vending machine agents to lower prices, purchase discounted inventory, and cancel orders from other users. OpenAI used this to illustrate that prompt injection issues do not merely affect chat outcomes but can also impact AI agents with execution capabilities.
Will still be used as an internal tool.
OpenAI states that GPT-Red is not currently available to the public, as the system intentionally includes trained offensive capabilities. The company positions it as a complement to internal red teams, third-party testing, and other security measures, rather than a replacement.
This also reflects a shift in the AI industry toward “using AI to protect AI.” Earlier this month, the Ethereum Foundation stated that it has deployed AI agents for red team testing of critical infrastructure, uncovering a vulnerability in Ethereum’s consensus client software. As model and agent capabilities improve, automated security testing is becoming an essential step before deploying AI systems.