AI compute bottlenecks are shifting from chips to packaging, interconnects, and storage.

iconKuCoinFlash
Share
AI summary iconSummary
In a recent Frictionless Podcast interview, Mr. Bubble of MetaEra noted that AI compute bottlenecks are now shifting from chips to packaging, interconnects, and storage. He argued that rising costs and diminishing returns in transistor density are making Moore’s Law increasingly unsustainable. The industry is turning to advanced packaging to build multi-chip systems rather than continuing to shrink process nodes. As AI models grow, he recommended using flash storage for historical data and HBM for frequently accessed data. Decentralized storage solutions could also help manage large volumes of data. Mr. Bubble emphasized that distributed storage will become a critical factor in system-level design, giving firms with expertise in packaging and storage a competitive advantage over those focused solely on large models.
ME AI News, Dongcha Beating AI Brief: Mr. Bubble, a seasoned chip designer and hardware analyst, stated in an interview on the Frictionless Podcast that the core bottleneck in AI compute infrastructure is shifting from merely increasing chip performance to advanced packaging, system interconnect latency, and memory hierarchy management. Mr. Bubble believes that the rising costs of advanced process nodes, coupled with diminishing gains in transistor density, are challenging the economic viability of traditional Moore’s Law. Rather than continuing to shrink process nodes, the industry is increasingly turning to advanced packaging to connect multiple chips into a single computing system. He noted that the fundamental unit of future AI compute may no longer be a single chip or server, but an entire rack—or even multiple racks—making PCB design, packaging, and interconnect capabilities critical limiting factors. Regarding memory, he argues that as AI inference context windows expand, vast amounts of historical context should not all occupy expensive HBM; instead, the industry is likely to adopt flash offloading, keeping high-frequency data in HBM while moving low-frequency context to larger, lower-cost storage layers. He further predicts that 3D DRAM may emerge as a key direction for next-generation memory technology. On AI infrastructure investment, Mr. Bubble believes that as pre-filling and decoding increasingly adopt disaggregated architectures, system-level hardware design will gain even greater importance. He also noted that companies with capabilities in hardware, packaging, storage, and other critical infrastructure may play a more enduring role in the AI supply chain compared to those directly betting on large model companies. (Source: BlockBeats)
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.