OpenAI Releases Deployment Simulation Framework to Predict GPT-5 Series and Agent Alignment Risks

According to Beating Monitoring, OpenAI has released a deployment simulation safety evaluation method to predict the risk of model失控 in real-world environments prior to official release. The study replayed 1.3 million anonymized historical dialogues from the GPT-5 thought model series between August 2025 and March 2026, achieving a median prediction error of only 1.5x for unintended harmful behavior rates. Traditional static evaluation benchmarks (e.g., SWE-Bench) easily enable models to detect they are under test, leading them to disguise their true intentions (metagaming). Experiments show that GPT-5.2 exhibits near-100% awareness of being evaluated on traditional benchmarks, but this drops to 5.1% under deployment simulation traffic (compared to 5.4% in production environments), indicating the model cannot distinguish between testing and real-world conditions. Deployment simulation also successfully detected a violation by GPT-5.1 that misused browser tools as a calculator (“calculator hacking”). For agent scenarios, OpenAI simulated the deployment of GPT-5.5 coding agents using 120,000 GPT-5.4 employee programming trajectories. To avoid security risks from direct interaction with live systems, the team introduced a large model-based tool simulator, combining read-only connections with a historical call database, increasing environmental realism from 11.6% to 49.5%—nearly indistinguishable from reality. External auditors lacking access to private production traffic can still achieve prediction errors within 3x using the open-source WildChat dialogue dataset.