Joseph Rodriguez, senior product marketing engineer for IP cores at Rambus, has written an article for Semiconductor Engineering that explores the company’s recent achievement of reaching 4 gigabits per second (Gbps) data rate with its HBM2E memory interface.
The milestone – which was demonstrated in silicon – required mastering substantial signal integrity and power integrity (SI/PI) challenges. To be sure, the 4 Gbps mark represents a 20% rise from the previous maximum data rate of 3.2 Gbps for HBM2E. To date, the industry’s fastest HBM2E DRAM comes from SK Hynix with operation of 3.6 Gbps.
Rambus, SK Hynix & Alchip Partnership
“Rambus teamed with SK Hynix and Alchip to implement the HBM2E 2.5D system to validate in silicon the Rambus HBM2E interface using TSMC’s N7 process and CoWoS (Chip-on-Wafer-on-Substrate) advanced packaging technologies,” Rodriguez explains. “Co-designing with the engineering team from Rambus, Alchip led the interposer and package substrate design.”
Paired with the 3.6 Gbps SK Hynix HBM2E DRAM, the Rambus solution can deliver 460 gigabytes per second (GB/s) of bandwidth. In an architecture with 4 to 6 HBM2E DRAM devices, this translates to two to three terabytes of bandwidth.
“This enormous bandwidth meets the needs of accelerators targeting the most demanding AI/ML training and high-performance computing (HPC) applications,” Rodriguez elaborates. “If you open the hood on the Rambus HBM2E memory interface, you’ll find a fully integrated and verified PHY and memory controller.”
As Rodriguez notes, the PHY design benefits from Rambus’ 30 years of high-speed signal expertise. In fact, it is thanks to that deep SI/PI design experience that Rambus achieved 4G operation over a 1024-bit wide interface. With command and address, there are over 1700 traces between the PHY on the SoC and the HBM2E DRAM.
“The pedigree of the Rambus HBM2E memory controller is also extremely impressive,” Rodriguez continues. “It is a second-generation design with a track record of implementation in over 50 customer designs and nine test chips. In addition, in every case [it has] been implemented with 100% first-time silicon success.”
Capacity requirements: Are they important?
As Rodriguez emphasizes, with AI/ML training models rising by 10X every year, capacity requirements are also a critical requirement for memory.
“The Rambus memory controller supports HBM2E DRAM with up to 12-high memory stacks. Further, the controller supports densities from 4 to 24 Gbits. With a 12-high stack and 24 Gbit density, that works out to 36 GByte per channel,” he adds.
According to Rodriguez, the controller is delivered configured to customer requirements to minimize size, power, and latency. The Rambus HBM2E solution also provides high bus efficiency across a wide variety of configurations (AXI, native interface) and traffic scenarios (random and sequential accesses; short and long bursts, etc.). In addition, it includes features for reliability, availability, serviceability (RAS) support – including ECC scrubbing and data path parity protection. Moreover, to ease in-lab bring-up or debug, the solution offers full featured memory test support to isolate the HBM2 Memory/PHY and Controller data path prior to full system bring-up.
“In addition to hardware-based validation, we also have an extensive factory-based verification environment using universal verification model (UVM). We use the Samsung and SK Hynix memory models, as well as Avery Design Systems memory models and monitors,” Rodriguez continues. You can be assured that you are not going to spend time debugging the PHY and controller interaction. We provide a customer focused version of this UVM testbench as part of every HBM2E controller delivery.”
As well, says Rodriguez, Rambus supports customers with industry-leading technical expertise during pre-sales and in all project phases including design, tapeout and bring up. Rambus’ Lab Station development environment further accelerates the bring up process.
“The rate of advancement in AI/ML is breathtaking with demands for greater bandwidth and capacity that are unrelenting. HBM2E delivers the highest bandwidth of any external memory solution by a wide margin, and Rambus offers the highest speed HBM2E interface solution. With our HBM2E memory interface, you can harness the highest level of performance using a proven, flexible and efficient memory solution backed up by world-class support,” he concludes.
Read our primer on:
HBM2E Implementation & Selection – The Ultimate Guide »