NVIDIA Blackwell Outperforms H200 by 20x in Agent Hardware Benchmark

ME AI News: According to monitoring by Beating, the evaluation agency Artificial Analysis has released the industry’s first agent hardware benchmark, AA-AgentPerf. Traditional evaluations are like a sprint—measuring only response speed—while agent tasks resemble a relay race, where AI must autonomously break down goals and repeatedly switch between reading/writing files, modifying code, and running tests. This high-frequency interaction places extreme demands on server memory capacity and scheduling efficiency. The benchmark replays real programming trajectories, using “concurrent agent capacity per megawatt of power consumption” as its core energy efficiency metric, directly addressing data center power and cost bottlenecks. The initial test ran the 1.6-trillion-parameter open-source model DeepSeek V4 Pro. Results showed that NVIDIA’s Blackwell liquid-cooled rack system, GB300 NVL72, can support 61,400 concurrent agents per megawatt, while the previous-generation Hopper HGX H200 supports only 2,600—representing a more than 20-fold improvement in energy efficiency. Single-GPU concurrent capacity also increased by 41 times. This enables data centers to support 20 times more concurrent agents under the same power budget, significantly reducing deployment costs for applications such as automated programming and customer service. Among the initial results, AMD Instinct MI355X currently lags behind. The evaluation agency noted that both AMD and H200 configurations were built using the general-purpose open-source vLLM framework without deep optimization; performance for AMD is expected to improve as service frameworks and internal kernel adaptations catch up. Currently, inference providers such as Together AI have already deployed DeepSeek V4 Pro on Blackwell systems to provide real-time inference support for the agent-based programming tool Cursor. (Source: BlockBeats)