Samsung and SK Hynix Diverge in HBM Design Strategies

At the Hot Chips conference, both Samsung and SK Hynitz agreed that GPUs, memory, foundry, and packaging must be designed as a system. Since then, their solutions have diverged.

Samsung is transforming its base chip into an active component within the system. It plans to move the memory controller from the GPU onto the logic base chip, freeing up 5–10% of GPU area for additional computation and potentially boosting performance by 10–20%. Selective AI operations can also run on the base chip to reduce data movement. In the long term, it is advancing zHBM technology, which directly stacks the GPU and DRAM, expected to reduce DRAM power consumption by approximately 70% while more than doubling bandwidth.

SK Hynix focuses on higher stacking while managing the resulting heat and warpage issues. It has confirmed that the HBM4 12-layer stack has entered production, while the 16-layer version is in customer validation. Hybrid bonding is the path to 20-layer and higher stacks, enabling the use of thicker dies and tighter pitches. Its iHBM approach adds a dedicated thermal conduction path within the hottest regions of the stack. This competition is no longer just about bandwidth and capacity—fundamental chip architecture and advanced packaging have become the decisive battlegrounds.

At the 2026 Hot Chips conference, Samsung Electronics unveiled zHBM, the ultimate evolution of high-bandwidth memory. This concept eliminates the 2.5D interposer, stacking DRAM directly atop compute chips such as GPUs to create a true 3D integrated architecture, aiming to reduce DRAM power consumption by 70% and achieve 2.3 times the bandwidth of HBM4E.

Samsung has outlined a three-phase roadmap to transform the base chip from a passive data transmission channel into an active computing partner. The first phase moves the memory controller from the xPU to the base chip to reclaim silicon area; the second phase adds memory expansion and attention computation capabilities to the base chip; the third phase ultimately achieves zHBM.

Next, let’s look at Samsung’s solution.

During the presentation, they shared some charts showing the trend of HBM5 specifications. While nothing is finalized yet, it's interesting to look at the chart in the upper right corner.

A 60GB HBM5 SSD with a bandwidth of 6 TB/s would be excellent. Let’s calculate the specifics: with a bandwidth of 6 TB/s and 2,048 channels, each pin would achieve a transfer rate of 23.5 Gbps. Our current cutting-edge speed is 16 Gbps, so this represents a significant advancement, likely achievable with advanced logic nodes. If each chip has a capacity of 3 GB and the total capacity is 60 GB, the stack height would be 20 layers.

On another slide, Samsung confirmed that the per-pin data rate for HBM4E is 16 Gbps. Our previous calculations for HBM5 have also been validated in the chart below.

Samsung’s clear advantage lies in its proprietary logic chips, whereas Micron and SK Hynix must rely on external partners to obtain such chips. Samsung highlights this strength and underscores the importance of customized chips. Samsung manufactures its memory using the 1C process and its logic chips using the 4nm process, significantly reducing power consumption. The diagram below satirizes Micron, which uses DRAM process technology for its logic chips in HBM4 memory.

Better yet, as mentioned earlier, Samsung has clearly divided its product roadmap into three phases, which we will cover one by one.

When logic nodes are present within the substrate, the physical area of the HBM PHY module decreases, and energy efficiency improves. This results in a smaller inter-chip module connecting the HBM and XPU. This offers two advantages:

Increase the area in XPU to perform more computations;

Increase the available area of the HBM substrate to enable more advanced features.

The drawback of reduced size is higher thermal density in these areas. Interestingly, they provide the actual dimensions of the PHY modules across different HBM generations.

Samsung also noted that they do not favor using UCIe in D2D links, as UCIe has a larger footprint and higher energy consumption per bit. They stated that their customized PHY implementation offers superior performance. In transitioning to HBM5, they demonstrated the concept of an Integrated Heat Path Block (HPB) designed to cool the D2D PHY area.

Another key advantage is the offloading of the memory controller to a custom base chip, freeing up space on the XPU chip that can be used to accommodate more floating-point processors.

The new advantage of integrating logic nodes on the substrate is the ability to implement SRAM. If a cell in the DRAM stack is damaged, the data in that cell can be temporarily stored in SRAM, and the memory controller will retrieve the data from SRAM instead of from the faulty DRAM cell. SRAM can be thought of as a backup notebook—a temporary notepad for storing data that cannot be saved in damaged DRAM cells.

Using logic nodes as the substrate means that if the transistor size is smaller, the silicon area required for many such functions is also reduced. The saved space can be used for various purposes, such as improving reliability, availability, and serviceability (RAS), conducting tests, and collecting telemetry data on the health and status of HBM memory stacks. Given that HBM telemetry data is the second leading cause of training failures, this data will be highly valuable.

If there is remaining silicon area, another can be added. The memory expansion controller is used to connect another HBM stack behind the first HBM stack or to insert DRAM chips to expand memory.

Finally, since logic transistors are available, why not also integrate some computational functions onto the substrate? The benefit is that computations such as matrix compression or encoding can be performed on the logic chip without leaving the substrate, offering advantages in both latency and efficiency.

If everything goes smoothly, it should theoretically be possible to build the intricate accelerator in the bottom-right corner. Drawing such a diagram is straightforward and conceptually intriguing, but whether it delivers significant real-world performance gains remains questionable.

The final stage involves using custom substrate chips and HBM, stacking them on top of the logic chip. The short distance between the compute and memory chips reduces the transmission distance for data bits, improving energy efficiency and increasing memory bandwidth by enabling more parallel data paths within the chip area—not just along the chip’s edge.

None of this is easy, but it’s crucial for the company to establish a roadmap for technical improvements. It demonstrates confidence in launching better products in the future, which is a positive sign for the company. Whether the market is ready to adopt this product is another matter entirely.

On August 23 at Hot Chips 2026, Jaesik Lee, Vice President of Packaging Engineering at SK Hynix America, stated that SK Hynix expects hybrid bonding technology to be insufficient for HBM4E, meaning the industry’s most anticipated transition in memory packaging will not occur until at least HBM5.

As he described, the issue lies in the total thickness of the HBM cube being limited to 775 micrometers—the standard thickness for 300mm logic wafers—so each additional DRAM layer requires thinner dies and narrower gaps. The 16-layer high-density HBM4 currently undergoing customer certification, which offers a single-cube capacity of 48GB compared to the already mass-produced 12-layer high-density HBM4, reduces the core die thickness to approximately 50 micrometers and halves the inter-die gap. Lee’s presentation also detailed the iHBM cooling architecture introduced three months after the company’s May release. Lee explained that this architecture has a limitation: the heat dissipation module cannot be applied to any HBM products already in design.

JED EC The HBM4 standard increases the maximum package thickness from 720 micrometers in HBM3E to 775 micrometers, alleviating the pressure caused by the adoption of hybrid bonding technology. Lee explained that when installing a cold plate on a GPU package, both the logic chip and the memory stack are partially ground down to expose bare silicon. Since the logic wafer thickness is 775 micrometers, any further increase in memory chip height would exceed the height of the adjacent processor. “This is the current limit we can achieve, because the logic wafer thickness is also 775 micrometers,” Lee said.

Thinner chips mean a higher proportion of oxide layers within the stack, and oxide layers have significantly lower thermal conductivity than silicon. Additionally, pin speeds have increased from 1 Gbps in early HBM to 8 Gbps in HBM4, resulting in higher power consumption per unit area. SK hynix’s own data shows that thermal load has increased on average by 2.2 times across the listed HBM generations, with the number of stacked layers doubling every two generations.

The company’s massive reflow mold underfill (MR-MUF) process, which picks and places all chips and connects them in a single reflow, has sacrificed profitability: according to Lee, filling gaps reduced by half while controlling warpage of chips smaller than 50 microns is the primary manufacturing challenge for 16-Hi.

Last May, Samsung publicly committed to using hybrid bonding technology for HBM4 production, while SK Hynix considered copper-to-copper bonding as an alternative for its advanced MR-MUF technology. Subsequently, JED EC The thickness constraints have been relaxed, making hybrid bonding no longer an urgent priority. Currently, the industry is discussing increasing the thickness of 20-layer stacks to 825 to 900 micrometers, which will again delay the adoption of copper bonding technology.

As early as March, industry insiders claimed that SK Hynix placed its first large-scale production order for hybrid bonding, a system worth approximately 20 billion Korean won ($150 million) that integrates tools from Applied Materials and Besi. Counterpoint Research expects this technology to enter full-scale HBM production around 2029 to 2030 with the launch of HBM5.

According to SK Hynix’s roadmap, hybrid bonding technology is currently still in the R&D phase and is designed for stack structures of 20 layers or more; SK Hynix has not yet decided which product will be the first to adopt this technology. Mr. Li did not disclose the specific generation, but excluding HBM4E, HBM5 is likely to be the first model to implement this technology. The technique connects flat copper pads and oxide surfaces at room temperature, then forms the bond using the thermal expansion of copper during the curing process.

“This seemingly simple process is actually highly challenging,” said Lee. “We need 16 or even 20 layers to achieve hybrid bonding, which is fundamentally different from single-layer stacking.” Removing microbumps entirely allows the core chip to be 24% thicker at a 20-layer height, reducing thermal resistance by approximately 35% compared to MR-MUF at the same height, with bump pitch (based on chip layer layout) below 18 micrometers, whereas MR-MUF currently has a pitch of 30 micrometers. At the bump pitch of HBM4, traditional microbumps remain viable, and each JED EC When increasing the thickness limit, MR-MUF can still maintain its feasibility and carry forward to the next generation of products.

The iHBM concept embeds thermally conductive, insulating blocks into the inter-chip PHY region of the base die—the interface hotspot with peak power density—claiming to reduce thermal resistance by more than 30%.

Lee’s slides directly compared iHBM with Samsung’s Heat Path Block solution. Samsung’s Heat Path Block uses dedicated heat dissipation pillars to extract heat from the chip stack, while Micron’s redesigned chip substrate circuitry claims a more than 20% improvement in energy efficiency. All three claims are vendor statements, each using different metrics. Both SK Hynix’s and Samsung’s designs are planned for HBM5, but neither is expected to enter mass production until after 2028. Lee noted that because these modules are located inside the package alongside the D2D PHY chip, they must be optimized according to customer designs and cannot be applied to already finalized chip generations, making iHBM a co-design effort. “This is a viable option we can pursue, but we cannot apply it to chip generations that have already been designed.”

During the Q&A session, Tanj Bennett from SemiAnalysis pointed out that stacking higher memory reduces the throughput of the silicon itself. DRAM operates at approximately 20 TB/s per square centimeter at the cell level; however, a 20-layer stack achieves a maximum throughput of about 4 TB/s, and HBM requires far greater manufacturing capacity than an equivalent DDR5 or LPDDR. “When stacked to 20 layers, this memory is actually slower on average than DDR5,” Bennett said. “Why not leverage the height of HBM stacks by placing cheaper memory around them instead?”

Li explained that training workloads require both bandwidth and capacity, and added that inference can balance both by keeping key-value caches in high-bandwidth memory while offloading part of the workload to LPDDR memory. This separation technique is already implemented in products such as NVIDIA’s Vera Rubin platform, which pools LPDDR5X and HBM4 via the NVLink-C2C protocol for this exact purpose. Additionally, SK Hynix and SanDisk have jointly developed a high-bandwidth flash memory specification that extends this hierarchical approach to NAND flash.

Lee commented on the layered solution: “This is also an issue we need to consider for the future.” According to reports in January, SK Hynix secured approximately 70% of the orders for NVIDIA’s Vera Rubin-generation HBM chips, all of which will utilize MR-MUF technology. Lee stated that the company has not yet decided which product will be the first to adopt MR-MUF technology.

During the speech, Micron will explore the evolving memory architectures in the field of artificial intelligence. Micron will detail how high-bandwidth memory and advanced packaging technologies are becoming central to AI system design.

Micron’s talk began with OpenAI’s frontier theory on computational efficiency, explaining why memory is critical for scaling language models. Micron noted a power-law relationship between model size, dataset size, and training compute, arguing that all three must scale in tandem to improve language model performance. Memory is the key resource that connects these three factors.

Now let’s discuss memory bottlenecks. Micron notes that the TFLOPS (floating-point performance) of AI accelerators roughly triples every two years, while the bandwidth growth of 2.5D stacked memory (such as HBM) is less than a doubling over the same period. This growing gap is precisely why memory technology has become a critical limiting factor in AI system performance.

Micron positions HBM as the bridge between computation and memory, tracking continuous improvements in capacity, bandwidth, and energy efficiency across generations from HBM2e to HBM5. You really need to appreciate this technical conference without Y-axis labels.

Here, Micron showcases a typical GPU with a package area exceeding 12,000 square millimeters, including the base chip and eight HBM4 memory instances. This means that when using 12 layers of HBM4 memory, the memory silicon area can exceed eight times the area of the GPU chip itself. This is indeed a very intuitive demonstration.

HBM increases the upper limit of memory bandwidth, shifting the boundaries of memory constraints and enabling memory-intensive AI workloads to run faster before reaching memory limits.

Micron's presentation is now providing a detailed explanation of what HBM is and the evolution of each HBM generation. The product table illustrates the progression from HBM1 to HBM4, with each generation increasing data rate, pseudo-channel count, and stack height to deliver higher bandwidth and greater capacity.

The installation method of HBM is the same as that of standard EC Different from C RDIMM. The diagram below shows how HBM stacks are integrated with GPUs or accelerators in heterogeneous integration systems and packaged as a system-in-package (SiP), rather than as standalone slot modules. Many people at STH have likely seen this design before.

Each DRAM chip contains multiple independent channels, each with two pseudo-channels; HBM3E supports 128 banks per chip, while HBM4 doubles this to 256 banks. The base die is positioned between the host and the DRAM stack, featuring a micro-bump PHY on the host side and a 3D TSV PHY on the stack side, connected via a high-density point-to-point interposer, increasing I/O count from 1K in HBM3E to 2K in HBM4.

Micron is currently showing the trade-off between performance and capacity. However, they haven’t truly demonstrated the differences in PCB area and power consumption here. Perhaps this will be shown in later slides?

Micron now recognizes the silicon cost associated with this speed. The combined costs of design architecture, advanced packaging, and manufacturing complexity mean that achieving the same HBM3E capacity as DDR5 requires approximately three times the amount of silicon.

Micron then summarized the memory parameters supporting AI innovation, including performance, capacity, and power consumption.

Higher-speed I/O interconnects and larger interposers continue to drive advancements in SiP technology, including faster I/O designs, memory-optimized SERDES and chip-to-chip PHYs, and co-packaged optics on the interconnect side. Larger SiPs based on technologies such as CoWoS-L and CoWoS-R, along with glass substrates, are also part of the evolution path for packaging technologies.

Now let’s discuss reliability, including chip-package interaction and RAS challenges. In heterogeneous integrated HBM devices, mismatched coefficients of thermal expansion between different materials generate thermomechanical stress. In fact, from a broader perspective, this is one of the biggest challenges faced in Cerebras’s WSE implementation. EC C coverage is divided into two layers: first, HBM3, where each 256-bit access uses 16 metadata bits; second, symbol-based Reed-Solomon implemented on-chip. EC C.

To manage heat, increased DRAM chip activity, higher stacking heights, and more advanced chip-level features all contribute to greater heat generation. Micron notes that liquid cooling, hybrid bonding, and improvements in solder and molding materials are essential thermal innovations required to sustain bandwidth and capacity scaling.

The packaging trend appears to be nearing its end.

Overall, these data illustrate how Micron is responding to the evolution of memory architectures in the field of artificial intelligence. Micron is balancing the benefits of increasing HBM bandwidth and capacity against the packaging, thermal, and reliability costs associated with placing such large amounts of memory adjacent to compute units.

What do people think HBM's price trend will be?

This article is from the WeChat public account "Semiconductor Industry Watch" (ID: icbank), authored by the editorial team.