Most people read NVIDIA's GPU lineup as a straight line: H100 fast, H200 faster, B200 fastest. Buy the newest thing you can afford.
That's one of the most expensive mistakes in AI infrastructure.
These three chips don't clear the same bottleneck harder. They clear three different ones:
The H100 is the workhorse. 80GB, strong compute. Its ceiling isn't math, it's capacity. 80GB.
The H200 is NOT a new chip. Same compute die as the H100. Identical FLOPS. NVIDIA only changed the memory: 141GB and 43% more bandwidth. Why? Because inference is bottlenecked by memory bandwidth, not compute. Result: about 40% faster serving for zero extra watts.
The B200 is the real jump. New architecture, dual-die, 192GB, 2.4x the bandwidth, native FP4, 1,000W. Roughly 2 to 2.5x an H100. But a different class of machine: new baseboards, new cooling, backlogged into mid-2026.
The rule that saves money: choose by bottleneck, not generation. Does the model fit (capacity)? Is decode starved for bandwidth? Do you actually need more compute? Buy the chip that clears the first wall you hit.
Most GPU regret is paying for peak compute when the workload was memory-bound the whole time.
#AI #GPU #AIModels