Microsoft’s Azure cloud platform just posted the fastest AI training results at the largest reported scale, powered by a deepened collaboration with Nvidia. The achievement, announced on March 18, 2025, centers on record-setting performance in the MLPerf Training v4.1 benchmarks, the widely recognized independent standard for measuring machine learning hardware performance.
The configuration behind the results: 512 Nvidia H200 GPUs working in concert, delivering a 28% performance improvement over previous setups built on H100 GPUs.
What the benchmarks actually show
In previous 2023 benchmarks, Azure showed it could train a GPT-3 model with 175 billion parameters on 10,752 H100 GPUs in approximately 4 minutes. The new H200-based configuration builds on that foundation with meaningfully better per-GPU performance, reducing the total hardware needed to hit comparable training speeds.
The full stack behind these results goes beyond just swapping in newer GPUs. Microsoft cited integrated innovations across hardware, networking, and software. The setup leverages Nvidia Quantum InfiniBand networking, which handles the massive data transfer demands between GPUs during distributed training. It also incorporates Nvidia’s microservices alongside Azure’s own AI services, including its AI Foundry platform.
Who’s already using this and what comes next
Black Forest Labs, the AI company known for its generative image models, is already utilizing Azure’s new GB200 virtual machines for its projects.
The GPU lineup supporting these capabilities spans multiple generations. Azure currently offers configurations built around the H200, H100, and GB200 series GPUs. Later in 2025, Microsoft plans to integrate Nvidia’s Blackwell Ultra GPUs, which are specifically designed to enhance performance in reasoning and multimodal AI workloads.