AI inference generates 130x more backend data than output, with storage costs often overlooked.

ME AI messages, when most executives imagine AI costs, they likely think first of GPUs—racks of accelerators consuming gigawatts of power—and the long waiting lists for scarce chips. But this image overlooks one thing: data storage costs.

After the model responds, what happens to the data? Billions of prompts, images, videos, and other backend data generated daily must be stored. However, data storage is one of the most challenging infrastructure costs to plan for and is often overlooked in initial AI infrastructure planning.
As the scale of reasoning expands, the data footprint also grows.
In the past, discussions around AI infrastructure focused primarily on the computationally intensive process of training models. Training is finite: a training job starts, consumes computational resources, and then ends. But inference runs continuously whenever a model responds to a prompt. This distinction is critical for infrastructure planning: training jobs are projects that can be budgeted for and then concluded; inference never ends, and each inference request leaves behind data.
Take AI-generated video as an example. In this type of workload, backend costs can be substantial. According to WD’s internal measurements, an 8-second video returned to the user as a 3.97MB 720p file generates over 500MB of associated backend data, including logs, cached contexts, real-time rendering outputs, backups, and security records—130 times the size of the output file the user sees. For enterprises generating longer, higher-resolution videos, the backend data footprint per video can scale to hundreds of gigabytes.
Moreover, this backend data footprint is growing rapidly. McKinsey estimates that by 2030, the compound annual growth rate (CAGR) for AI inference demand will reach 35%, higher than the 22% for training demand; by the end of this decade, inference will account for more than 40% of total data center demand.
A recent IDC global white paper sponsored by WD surveyed 763 IT and business decision-makers. The survey found that nearly 95% of respondents reported that generative AI led to an increase in data volume over the past 12 months; approximately 61% indicated that the increase in data volume reached or exceeded 25%.
You can think of increasing storage as an optimization to keep the GPU busy,” the report quotes a senior product manager at a hyperscale cloud provider as saying. “The biggest challenge is ensuring the GPU doesn’t sit idle, waiting for data to be ready—GPU idleness for hours can become extremely expensive very quickly.” In dollar terms: if storage spending grows by 25% annually, while compute budgets grow in more predictable, cyclical patterns, storage will eventually surpass that most closely watched budget line.
Align storage with the AI data lifecycle
Computational demand fluctuates with training cycles, product releases, and hardware updates. However, when computational demand slows, the volume of data retained does not decrease. Data such as model logs, synthetic outputs, and inference results continue to accumulate, and organizations are retaining more data for longer periods.
This has changed the role of retained data. Information that was previously moved to archives and rarely accessed may now serve as input for future AI workloads. As more data is retained, retrieved, and reused, storage must not only support growing capacity but also enable increasingly dynamic access to this data. The same WD-IDC survey shows that 96% of respondents expect faster archive retrieval capabilities to support inference or retrieval-augmented generation (RAG).
AI workloads require both performance and capacity, and different stages of the data lifecycle demand different technologies to meet specific needs. For large volumes of data that must be retained and frequently accessed, HDDs provide a cost-effective, high-capacity foundation with significant advantages in cost per TB.
What does this mean for AI budgeting?
For a CFO or CIO, this means the scope of "AI infrastructure spending" may be too narrow, overly focused on compute. When a model sits idle while loading data, its cost is roughly equivalent to sitting idle due to a lack of GPUs. However, storage bottlenecks tend to remain hidden until the problem has already emerged; they typically manifest as increased operational costs and retrieval latency, rather than appearing directly in capital expenditure (CapEx) projects like accelerators do.
Imagine a retailer generating AI product videos for its catalog, or an insurance company processing claims documents via a RAG workflow to accelerate underwriting. In both cases, compute costs are known and budgeted items: the finance team can forecast expenses based on per-generation or per-query rates. Storage costs, however, are different: each video generated or each claims document processed adds to the cost, because the backend data does not disappear after the task is completed.
If a team sets their AI budget at “computational costs plus a negligible storage fee,” they may find, 12 to 18 months later, that this so-called “negligible fee” has become a significant portion of the project’s total cost—just as management is preparing to decide whether to scale the project.
Build capacity with economic scalability
Addressing this challenge requires a structural approach. Storage strategies are often designed around the high cost of hot-layer performance to meet immediate, continuous data access needs; however, in terms of volume, most AI data accumulates in the subsequent warm, cold, and deep-cold storage layers.
Enterprises need capacity with economic scalability: infrastructure designed for the full AI data lifecycle, balancing performance, capacity, power, reliability, and total cost of ownership as data persists, accumulates, and is increasingly reused. This means treating previously long-idle archived data as active assets for RAG and inference, and formulating data retention policies based on the evolving strategic value of data rather than its original storage cost. It also means recognizing that storage decisions impact the entire storage cluster, not individual hard drives.
This type of infrastructure requires layered storage. During capacity expansion, the power consumption, reliability, storage density, and operational efficiency of each layer collectively determine whether the system can maintain economic sustainability.
Regarding the organization's AI infrastructure budget, three questions can be asked:

What is our data retention policy for AI-generated content, and who established it?
Is our storage procurement based on performance requirements during training, or on the actual lifecycle of the generated data?
If our AI usage doubles next year, will storage costs double as well, or increase even faster?

Most organizations cannot confidently answer these three questions. This is precisely the gap that should be addressed before the next budget cycle, rather than waiting until problems arise.
This content was created by WD and was not written by the MIT Technology Review editorial team. (Source: MLion)