Amazon Web Services, the cloud computing giant that powers a startling chunk of the internet, is doing something that sounds almost quaint for a trillion-dollar company: asking its engineers to turn off computers they’re not using.
The directive targets idle EC2 instances, the virtual servers that form the backbone of AWS’s cloud infrastructure. With AI workloads devouring GPU capacity at an accelerating rate, even the world’s largest cloud provider is feeling the squeeze.
The scale of the waste problem
The numbers paint a picture of staggering inefficiency. Roughly 65% of EC2 instances maintain average CPU utilization below 20% over 30-day measurement windows.
AWS’s Compute Optimizer tool has been upgraded to catch these ghost servers. It now analyzes CPU utilization and network I/O over 14-day lookback periods, flagging instances where peak CPU stays below 5% and network traffic is negligible.
Amazon CEO Andy Jassy disclosed that AWS added over 3.8 gigawatts of power capacity in the previous year. For context, that’s enough electricity to power roughly 2.8 million homes. The company plans to double its capacity by 2027.
AI demand is breaking the economics
The capacity crunch has a clear driver: artificial intelligence. GPU-accelerated instances, the workhorses behind training and running large language models, have become the scarcest resource in cloud computing.
This demand pressure has shown up directly in pricing. EC2 Capacity Blocks for machine learning saw a roughly 15% price increase in January, followed by another hike of approximately 20% by July. Two price bumps in six months for the same service reflects the supply-demand imbalance.
The persistent GPU shortage has reportedly pushed some AWS customers to consider shifting operations to competing cloud providers.
What this means for the cloud market
For businesses running workloads on AWS, the pricing trajectory is worth watching closely. Two significant price increases on ML capacity blocks within a single year suggest that companies heavily reliant on cloud-based AI processing should be budgeting for continued cost escalation.
The 65% underutilization figure isn’t just an AWS problem. It reflects industry-wide habits of over-provisioning resources because spinning up a new instance has always been easier than right-sizing an existing one.
Amazon’s plan to double its data center capacity by 2027 represents one of the largest infrastructure buildouts in corporate history. But between now and then, the company appears to be taking a belt-and-suspenders approach: build more capacity while simultaneously squeezing more utility out of what already exists.