HBM4 Redefines AI Hardware with System-Level Innovations

As demand for AI large model training and inference continues to surge, the performance bottleneck of compute chips has shifted from computational units to memory bandwidth, making HBM (High Bandwidth Memory) the critical constraint on the advancement of high-end AI hardware. At the Hot Chips 2026 international conference on cutting-edge chip technology, the world’s two leading memory manufacturers, Samsung and SK Hynix, jointly revealed the next-generation evolution roadmap for HBM4, breaking the industry’s long-standing HBM iteration logic.

Compared to previous generations that focused solely on increasing DRAM speed and stacking layers, the new HBM4 architecture fully shifts toward three innovative directions: upgrading the base die logic process, implementing hybrid bonding 3D stacking, and adopting EMIB heterogeneous hybrid packaging. Behind this technological evolution is HBM’s transformation from a standalone memory device into an integrated system encompassing memory, logic, packaging, IP, and EDA.

During the previous iterations of HBM3 and HBM3E, the industry’s technical upgrade logic remained relatively consistent, with core optimizations focused primarily on the DRAM memory dies themselves—achieving steady increases in overall bandwidth through higher per-die data transfer rates, increased stacking layers, and improved micro-bump interconnect structures. Under this architecture, the bottom Base Die served only passive roles in signal routing, power distribution, and test assistance, acting merely as a signal bridge. The process architecture long relied on mature, storage-derived processes, requiring no major iterations; as a result, the entire technological barrier and production bottleneck for HBM products were entirely concentrated in the DRAM manufacturing stage.

However, as AI computing demand grows exponentially, the marginal returns of traditional iterative approaches are rapidly diminishing. Under ultra-high-bandwidth scenarios, simply increasing DRAM speed leads to severe signal crosstalk, soaring power consumption, and timing skew—optimizations on a two-dimensional plane can no longer meet the demands of such massive computational loads. To address this industry challenge, at this year’s HotChips 2026 conference, Samsung and SK Hynix simultaneously unveiled new technical directions, clearly identifying that the core breakthrough in the HBM4 era is no longer the memory die itself, but the systematic redesign of the underlying base die.

The two major manufacturers have adopted clearly differentiated technical approaches. At Hot Chips, Samsung disclosed that its HBM4/HBM4E products use its proprietary 4nm logic process for the Base Die and a 1cDRAM node for the CDie, achieving a maximum pin rate of 11.7 Gbps, scalable up to 13 Gbps, with a single-stack bandwidth of up to 3.3 TB/s. Leveraging the high transistor density of its logic process, the Base Die integrates higher-performance PHY circuits, signal error correction modules, and power management units, significantly improving high-speed signal integrity and resolving timing disorders under ultra-high bandwidth conditions. Meanwhile, Samsung’s architecture design explicitly moves certain memory control logic to the Base Die, effectively freeing up silicon area on the GPU side and creating space for expanding compute units.

SK Hynix adopts a collaborative, iterative approach by partnering with TSMC, using TSMC’s 12nm logic process for the Base Die. Compared to Samsung’s fully in-house logic architecture, SK Hynix’s solution prioritizes manufacturing stability, optimizing the power delivery network and thermal conduction structure to meet the cooling and load demands of 12-layer and even 16-layer ultra-high stacking architectures. This differentiated strategy means that the internal logic architecture of HBM4 Base Dies is no longer uniform; products from different vendors naturally vary in process, performance, and application suitability, introducing new complexities for subsequent supply chain integration and chip design iterations.

Switching the Base Die to a logic process technology delivers multiple benefits. Logic processes enable higher transistor density, optimize signal integrity and power consumption of PHY circuits, and support the doubled 2048-bit I/O width following HBM4. Meanwhile, the additional chip area on the Base Die provides hardware space for near-memory computing, allowing HBM to function not merely as a data container but also to perform basic data preprocessing internally, alleviating the data movement pressure inherent in the von Neumann architecture. However, the costs are equally significant: HBM is no longer a product that DRAM manufacturers can fully close-loop produce. Logic foundry capacity, IP design capabilities, and cross-process co-simulation all become critical constraints for HBM mass production. DRAM vendors must deeply collaborate with logic foundries, extending design, verification, and tape-out cycles while exponentially increasing supply chain complexity.

Another key signal from HotChips2026 is that the system integration packaging roadmap for HBM is becoming more diversified—TSMC’s CoWoS is no longer the only solution, as hybrid packaging approaches enter the stage, and internal stacking bonding technology transitions toward hybrid bonding 3D stacking.

SK Hynix confirmed in its presentation that it is evaluating Intel’s EMIB embedded silicon bridge technology for 2.5D system integration between HBM4 and compute chips. Traditional CoWoS relies on a large-area silicon interposer to connect all signals between the GPU and HBM, offering excellent performance but at a high cost. Additionally, production scaling is constrained by the limited size of photomask tools, leading to chronic global supply shortages for CoWoS. Leading customers have secured priority access to capacity, forcing other manufacturers to wait extended periods for production slots. EMIB eliminates the need for a full silicon interposer, instead embedding small silicon bridges only where chips require high-density signal interconnects, while the majority of routing is handled by an organic substrate. This approach reduces packaging costs, overcomes photomask size limitations, and enables larger-scale multi-chip integration. Google’s TPU has already included EMIB in its selection list for next-generation designs.

However, EMIB does not mean a direct replacement for CoWoS. The localized silicon bridge in EMIB still lags behind full silicon interposers in terms of signal loss and power integrity under extremely high-density signal scenarios. SK Hynix’s strategy is to position EMIB as an important complementary pathway, not a complete replacement. High-end training chips will continue to prioritize CoWoS, while mid-to-high-end inference and high-compute accelerators can adopt EMIB solutions to alleviate packaging capacity pressures, creating a dual-path approach.

Within HBM stacks, the bonding technology between multiple layers of DRAM dies is currently transitioning from MRMUF to hybrid bonding. For the current mass-produced HBM4 versions, the mainstream solution still employs MRMUF and TCNCF thermocompression bonding, relying on microbumps for inter-layer interconnects. However, as stack heights increase toward 16 layers, the pitch, thermal resistance, and power consumption of microbumps are gradually reaching physical limits. Compared to MRMUF, hybrid bonding not only increases the core chip thickness by 24% and reduces TSV pitch to below 18 micrometers, but also significantly lowers thermal resistance by 35% while enabling higher stack layers. This technology is expected to first appear in HBM5, targeting ultra-high stacks of 16 to 20 layers or more.

However, hybrid bonding is currently in the small-volume validation stage, with challenges in yield control, equipment costs, and thin wafer handling. Widespread commercial adoption is not expected until 2027–2028, when the HBM4E iterative version and the HBM5 generation are anticipated to enable large-scale implementation of hybrid bonding.

This has created a uniquely transitional technological landscape in the current HBM industry: for the next-generation HBM4 product, external system integration is pioneering diverse EMIB heterogeneous packaging solutions to alleviate pressure on high-end packaging capacity, while internal DRAM stacking continues to rely on the mature MRMUF thermocompression process to ensure production stability, with hybrid bonding only in preliminary R&D stages. The simultaneous advancement of multiple technical pathways and the coexistence of old and new processes have significantly increased uncertainty in HBM4 technology selection at this stage.

In addition, thermal management has become an unavoidable core challenge for ultra-high-stack HBM. The 16-layer stacking architecture significantly increases the overall thermal load of HBM; heat accumulation directly leads to bandwidth throttling and inability to achieve peak performance. To address this pain point, two manufacturers have developed differentiated solutions: SK Hynix has introduced an IHBM solution with integrated thermal components, embedding highly conductive and electrically insulating cooling elements within the HBM’s D2D PHY region to create a dedicated thermal path, reducing thermal resistance by over 30% and enhancing system operational stability; Samsung, on the other hand, is planning copper thermal via designs for its future HBM5 products, tackling the thermal challenges of ultra-high stacking at the hardware architecture level. It is clear that the performance ceiling of future high-stack HBM products will no longer be determined solely by bandwidth and speed parameters, but will be directly dictated by thermal management capabilities.

Today, the core barriers of HBM4 extend beyond manufacturing and packaging to include two critical software capabilities: high-speed IP and heterogeneous EDA toolchains—areas that are currently the most overlooked yet most crucial bottlenecks in the industry. As HBM4 interface speeds surpass 9 Gbps and IO width doubles, the design complexity of high-speed PHY and SerDes interface IPs has increased exponentially.

High-speed IP is not a simple circuit design; it requires deep adaptation to DRAM chip characteristics, Base Die logic architecture, and packaging layout structure to achieve joint simulation and debugging for signal and power integrity. Currently, mature, mass-producible high-speed IP for HBM4 is highly concentrated among a few overseas leading vendors, who dominate the IP supply market for mainstream computing chips through years of fabrication experience. For mid- and small-sized computing design companies and emerging supply chain manufacturers, inability to access mature high-speed IP prevents them from adapting to HBM4 architecture, creating a stringent industry entry barrier.

Meanwhile, the shortcomings of the 3D heterogeneous EDA toolchain further increase the difficulty of implementing HBM4. HBM4 is a typical multi-process, multi-die heterogeneous integration system, with the logic substrate, memory stack, and silicon bridge/interposer packaging belonging to distinct process technologies. Traditional 2D standalone EDA tools cannot perform cross-die, cross-process co-simulation. In 3D stacking scenarios, electrical, thermal, and mechanical effects are intercoupled; simulation inaccuracies in any single component can lead to overall design convergence failure.

The current industry-standard design process suffers from significant fragmentation: frontend logic design, mid-end physical implementation, and backend packaging and simulation data cannot interoperate or be reused, resulting in extremely high iteration costs. Hot Chips 2026 explicitly states that the large-scale deployment of HBM4 heavily depends on an integrated 3DIC collaborative design toolchain that enables seamless coordination across design, process, packaging, and multi-physics simulation. The adaptability of EDA tools and the debug compatibility of high-speed IPs directly determine HBM4’s manufacturing yield and performance ceiling, representing the core technological barriers that small and medium-sized companies currently struggle to overcome.

Overall, this year’s conference fully outlines a new evolution path for the HBM industry: moving away from the traditional model of isolated hardware upgrades and entering an era of systemic competition characterized by coordinated advancements in architecture, packaging, manufacturing processes, and software-hardware ecosystems. The differentiated technological approaches of Samsung and SK Hynix have disrupted the industry’s long-standing standardization framework. Coexistence of legacy and advanced processes, layered application-specific technologies, and system-level capabilities as decisive factors will become the core developmental norms for the HBM sector over the next two to three years, rewriting the rules of competition in the AI storage industry.

This article is from the WeChat official account "Semiconductor Industry纵横" (ID: ICViews), author: Zihao.