Enter High Bandwidth Memory 4 (HBM4), a 3D-stacked memory technology that promises unprecedented bandwidth per chip. This could determine which companies dominate – or disappear from – the AI landscape. This isn’t just another incremental upgrade; it’s the difference between training the next breakthrough AI model in weeks versus months, between profitable inference and burning cash with every query.

I’d think that the limiting factor would be the memory controller and bus, not the memory chips themselves. It’d be less cost-efficient, maybe, but I can just throw more physical memory chips into a server to buy more bandwidth at the memory level.

You can only get so much memory so physically close to a processor, and that places latency constraints, but I don’t believe that the parallel-heavy compute from AI workloads is particularly latency-sensitive, unlike many traditional workloads.

Samsung’s troubles become evident with HBM3E. While SK hynix and Micron ramped 8-high and 12-high HBM3E for customers, Samsung struggled to get its 12-high stacks qualified. It reportedly took 18 months and several attempts to meet Nvidia’s quality and performance criteria for HBM3E. By Q3 2025, Samsung had finally cleared Nvidia’s validation, with its 5th-generation HBM3E 12-stack passing all tests.

Until now, Samsung HBM has appeared only in AMD’s MI300-series accelerators. However, with Nvidia’s certification, the company has agreed to purchase between 30,000 and 50,000 units of 12-high HBM3E for use in liquid-cooled AI servers. Samsung’s HBM3E is also shipping for AMD’s accelerators as of mid-2025.

A bit reminiscent of their fight against TSMC in the semiconductor fabrication space. If Intel does have return to form (which is a big if), Samsung might once again fight themselves in third place.