Server DIMM Chipsets

– Last updated on: October 17, 2019 – 

With the upcoming industry transition to DDR5 server dual-inline memory modules (DIMM), there are a number of key performance and power gains on the horizon, as well as new design challenges. Server system architects and designers want to know what’s new in DDR5 and how can they get the most from this new generation of memory.

In this article:

• • What changes in DDR5 vs. DDR4?
  • What are the DDR5 design challenges?
  • How do DDR5 memory interface chipsets harness the advantages of DDR5 for DIMMs?

What changes in DDR5 vs. DDR4?

The top six most significant specification advances made in the transition from DDR4 to DDR5 DIMMs are shown in Table 1 below.

1. DDR5 Scales to 6,400 MT/s

You can never have enough memory bandwidth, and DDR5 helps feed that insatiable need for speed. While DDR4 DIMMs top out at 3,200 megatransfers per second (MT/s) at a clock rate of 1.6 gigahertz (GHz), that’s where DDR5 will start. With initial designs likely at 4,800 MT/s, DDR5 can scale further to double the data rate of DDR4 reaching 6400 MT/s.

2. Lower Voltage Means Lower Power

A second major change is a reduction in operating voltage (VDD), and that will translate to lower power. With DDR5, the DRAM and buffer chip registering clock driver (RCD) voltage drops from 1.2 V down to 1.1 V. However, lower VDD means smaller margin for noise immunity which designers will have to be cognizant of for their implementations.

3. New Power Architecture for DDR5

A third change, and a major one, is power architecture. With DDR5 DIMMs, power management moves from the motherboard to the DIMM itself.  DDR5 DIMMs will have a 12-V power management IC (PMIC) on DIMM allowing for better granularity of system power loading. The PMIC distributes the 1.1 V VDD supply, helping with signal integrity and noise with better on-DIMM control of the power supply.

4. DDR5’s New Channel Architecture

Another major change with DDR5, number four on our list, is a new DIMM channel architecture. DDR4 DIMMs have a 72-bit bus, comprised of 64 data bits plus eight ECC bits. With DDR5, each DIMM will have two channels. Each of these channels will be 40-bits wide: 32 data bits with eight ECC bits. While the data width is the same (64-bits total) having two smaller independent channels improves memory access efficiency. So not only do you get the benefit of the speed bump with DDR5, the benefit of that higher MT/s is amplified by greater efficiency.

In the DDR5 DIMM architecture, the left and right side of the DIMM, each served by an independent 40-bit wide channel, share the RCD. In DDR4, the RCD provides two output clocks per side. In DDR5, the RCD provides four output clocks per side. The 32-bit data of each 40-bit channel consist of four 8-bit lanes, and each of these lanes gets an independent clock signal from the RCD. Giving each lane an independent clock improves signal integrity, helping to address the lower noise margin issue raised by lowering the VDD (from change #2 above).

5. Longer Burst Length

The fifth major change is burst length. DDR4 burst chop length is four and burst length is eight. For DDR5, burst chop and burst length will be extended to eight and sixteen to increase burst payload. Burst length of sixteen (BL16), allows a single burst to access 64 Bytes of data, which is the typical CPU cache line size. It can do this using only one of the two independent channels. This provides a significant improvement in concurrency and with two channels, greater memory efficiency.

6. DDR5 Supports Higher Capacity DRAM

A sixth and final change to highlight for DDR5 is support for higher-capacity DRAM devices. With DDR5 buffer chip DIMMs, the server or system designer can use densities of up to 32-Gb DRAMs in a single-die package. DDR4 maxes out at 16 Gb DRAM in a single-die package. DDR5 supports features like on-die ECC, error transparency mode, post-package repair, and read and write CRC modes to support higher-capacity DRAMs.

What are the DDR5 Design Challenges?

These changes in DDR5 introduce a number of design considerations dealing with higher speeds and lower voltages – raising a new round of signal integrity challenges. Designers will need to ensure that motherboards and DIMMs can handle the higher signal speeds. When performing system-level simulations, signal integrity at all DRAM locations need to be checked.

For DDR4 designs, the primary signal integrity challenges were on the dual-data-rate DQ bus, with less attention paid to the lower-speed command address (CA) bus. For DDR5 designs, even the CA bus will require special attention for signal integrity. In DDR4, there was consideration for using differential feedback equalization (DFE) to improve the DQ data channel. But for DDR5, the RCD’s CA bus receivers will also require DFE options to ensure good signal reception.

The power delivery network (PDN) on the motherboard is another consideration, including up to the DIMM with the PMIC. Considering the higher clock and data rates, you will want to make sure that the PDN can handle the load of running at higher speed, with good signal integrity, and with good clean power supplies to the DIMMs.

The DIMM connectors from the motherboard to the DIMM will also have to handle the new clock and data rates. For the system designer, at the higher clock speeds and data rates around the printed circuit board (PCB), more emphasis must be placed on system design for electromagnetic interference and compatibility (EMI and EMC).

How do DDR5 memory interface chipsets harness the advantages of DDR5 for DIMMs?

The good news is that DDR5 memory interface chips improve signal integrity for the command and address signals sent from the host memory controller to the DIMMs. The bus for each of the two channels goes to the RCD and then fans out to the two halves of the DIMM. The RCD effectively reduces the loading on the CA bus that the host memory controller sees.

DDR5 data buffer chips will reduce the effective load on the data bus, enabling the higher-capacity DRAMs on the DIMM without degrading latency.

Rambus offers a DDR5 memory interface chipset that helps designers harness the full advantages of DDR5 while dealing with the signal integrity challenges of higher data, CA and clock speeds.

As a renowned leader in signal integrity (SI) and power integrity (PI), Rambus has a 30 year of history in enabling the highest performance systems in the market.

Earlier this month, Semiconductor Engineering’s Ann Steffora Mutschler penned an article that takes a closer look at how buffering is gaining ground as a way to speed up the processing of increasingly large quantities of data. In simple terms, says Mutschler, a data buffer is an area of physical memory storage that temporarily stores data while it is being moved from one place to another.

“This becomes increasingly necessary in data centers, autonomous vehicles and for machine learning applications,” she explains. “The challenges with these applications are advanced signal equalization and increased capacity and bandwidth. Data buffering techniques — either as a discrete chip within the memory module or integrated into an SoC — help make this all possible.”

As Mutschler notes, buffer chips manage read and write commands from the CPU to the DRAM. When buffer chips are used, it is known as a load reduced dual inline memory module (LRDIMM). Those chips boost total memory capacity at the same performance and speed.

“As data is growing at exponential rates, reducing CPU load through data buffer chips becomes necessary to increase DRAM capacity per CPU,” Victor Cai, director of product marketing for Rambus‘ Memory and Interfaces Division, told the publication. “LRDIMMs are common in data centers that are performing data-intensive applications, such as big data analytics, artificial intelligence and machine learning.”

As Cai noted in a previous Semiconductor Engineering article, DDR3 server DIMM chipsets (800 Mbps) first hit the market in 2006 and began to ramp the following year. By the time DDR4 server DIMM chipsets (2133) began shipping in 2014, DDR3 server DIMM chipsets were spanning the following five speeds: 800, 1066, 1333, 1600 and 1866. In the last years, DDR4 buffer chipset shipments have crossed over in term of volume, with DDR4 chipset speeds expected to reach 3200 by 2018 or 2019.

In addition to increased bandwidth and density, DDR4 buffer chips offer data centers the critical benefits of advanced signal equalization. For example, Rambus registered clock driver (RCD) and data buffer (DB) chips are equipped with sophisticated decision feedback equalizers (DFEs) which are typically used in the communication industry for 10G or higher SerDes. Essentially, DFE circuits allow RCDs and DBs to operate with greater margin on typical systems – as well as in challenging environments where older chipsets are prone to malfunction or fail to achieve higher speeds.

Equipping both chips (RCD and DB) with DFE capabilities allows the DIMM chipset to push the limits of both the command address bus as well as the DQ bus. Moreover, DFE circuits enable DIMMs to run at speeds of 3200, exceeding original industry expectations for this generation of buffer chips. It is also important to emphasize that signal integrity challenges are becoming more prevalent as bandwidth, density and speeds increase. Fortunately, lessons learned from other markets (such as communications) will merge with DDR technology to help push the single-ended bus to its limit.

From our perspective, memory buffer chips will continue to steadily evolve, as is illustrated by the number of technical advances (such as DFE) seen from one generation of silicon to the next. The importance of server DIMM chipsets will only continue to increase, as the industry attempts to sustain Moore’s Law and maximize the limitations of Von Neumann architecture in a changing data center.

Interested in learning more about data buffering? The full text of “Data Buffering’s Role Grows” by Ann Steffora Mutschler can be read on Semiconductor Engineering here.

Victor Cai, a director of product marketing at Rambus, recently wrote an article for Semiconductor Engineering that explores the evolution of server DIMM chipsets in the data center.

According to Cai, DDR3 server DIMM chipsets running at 800 Mbps first hit the market in 2006 and started to ramp in 2007. 7 years later, DDR4 server DIMM chipsets running at 2133 began shipping. Concurrently, DDR3 server DIMM chipsets spanned the following five speeds in 2014: 800, 1066, 1333, 1600 and 1866.

“In the last years, DDR4 buffer chipset shipments have crossed over in term of volume, with DDR4 chipset speeds expected to reach 3200 by 2018 or 2019,” Cai explains. “In addition to increased bandwidth and density, DDR4 buffer chips offer data centers the critical benefits of advanced signal equalization.”

As an example, says Cai, Rambus registered clock driver (RCD) and data buffer (DB) chips are equipped with sophisticated decision feedback equalizers (DFEs) which are typically used in the communication industry for 10G or higher SerDes.

“Essentially, DFE circuits allow RCDs and DBs to operate with greater margin on typical systems – as well as in challenging environments where older chipsets are prone to malfunction or fail to achieve higher speeds,” states Cai. “Equipping both chips (RCD and DB) with DFE capabilities allows the DIMM chipset to push the limits of both the command address bus as well as the DQ bus.”

Moreover, says Cai, DFE circuits enable DIMMs to run at speeds of 3200, exceeding original industry expectations for this generation of buffer chips. However, signal integrity challenges are becoming more prevalent as bandwidth, density and speeds increase.

“Fortunately, lessons learned from other markets (such as communications) will merge with DDR technology to help push the single-ended bus to its limit,” he notes.

With regards to current DIMM chipset deployment, Cai observes that DDR4 buffer chipset shipments are ramping up as the price of DDR4 silicon continues to decrease. This, says Cai, is because data centers only upgrade their memory subsystems when the economics of performance per dollar meets a certain threshold.

“For most data centers, this means a transition from DDR3 to DDR4 within the DDR4-2400 generation – as price parity is reached between DDR3 DRAM and DDR4 DRAM,” he elaborates. “Many industry heavyweights have already transitioned or are in the process of switching over to DDR4 as their primary server memory type. Looking towards the near future, speed increases and price reductions will further accelerate adoption of DDR4 server DIMM chipsets by data center customers.”

In conclusion, says Cai, memory buffer chips continue to steadily evolve, as is illustrated by the number of technical advances (such as DFE) seen from one generation of silicon to the next.

“The importance of server DIMM chipsets will only continue to increase, as the industry attempts to sustain Moore’s Law and maximize the limitations of Von Neumann architecture in a changing data center,” he adds.

DDR5

Ann Steffora Mutschler recent penned an article for Semiconductor Engineering that explores the role of new memory technology in the evolving data center. For example, DDR5 is expected to offer significant improvements over previous iterations of DRAM, including doubling the bandwidth and density over DDR4, as well as facilitating optimized channel efficiency. Nevertheless, DDR5 cannot stand alone as the industry struggles to meet the tsunami of data generated by an increasingly connected world.

“The amount of data being generated by sensors in a variety of connected devices across multiple markets is exploding and so is the demand for compute power to make sense of [it] all,” Mutschler explained. “Machine learning/AI/deep learning, separately and as extensions of markets filled with connected devices, are only adding to the need for additional types of memories.”

DRAM scaling slows

In addition to DDR5, the industry is eyeing GDDR5 (originally targeted at the graphics market), HBM2 and NVDIMM (persistent memory). This is because no single memory type is capable of satisfying the insatiable demand for ever more bandwidth and capacity, especially as Moore’s Law wanes and DRAM scaling slows.

“It’s been difficult for large memory manufacturers like Samsung, SK Hynix and Micron to get the densities higher and higher. Right now, 4- and 8-gigabit devices are shipping in mass production in high volumes,” Sarvagya Kochak, a senior product marketing manager at Rambus, told Semiconductor Engineering. “16 gigabit should be on the market in a year or two, but this poses a pretty significant challenge because if you look at it from a CPU perspective, CPU speeds haven’t gone up over the last decade. They have more or less leveled off because transistor speeds cannot be bumped up more than they are right now.”

To compensate, says Kochak, CPU vendors are adding more cores inside of processors, although this presents its own set of challenges.

“The problem with this approach is that within the CPU is an integrated memory controller, which interfaces with the DRAM DIMMs. With older CPUs, there would have been two memory controllers. Now there are four memory controllers. Soon there will be six memory controllers and the projection is that there may even be eight memory controllers per CPU,” he elaborated. “The reason why there are more memory controllers is because there needs to be more data to feed all of the cores, so if there are fewer memory controllers, the data can’t be fed from the DRAM to the cores. If the cores are hungry, there’s no point in having them. As the number of cores goes up, there is a requirement for more bandwidth from the DRAM into the system. If you look at the rate at which memory has been increasing, as well as the memory attach rate per CPU, it’s not a pretty picture.”

NVDIMMs (persistent memory)

As Kochcak previously explained in a detailed Semiconductor Engineering article, the above-mentioned DRAM challenges have prompted system and software architects to rethink standard memory-storage subsystems in an effort to extract all the performance they possibly can with currently available technologies. More specifically, the majority of application, OS and software stacks are (presently) designed with the underlying assumption that memory is volatile and the contents in DRAM can be lost. Complex mechanisms ensure little to no data is lost in the event of a power failure. Although this comes at the cost of performance, the industry has learned to accept the tradeoff.

With non-volatile DIMMs (NVDIMM) technology, it is possible to accelerate system performance by making memory persistent and eliminating the aforementioned tradeoff. As Mutschler observes, NVDIMMs operate as standard DRAM, while also offering the persistence of flash. A typical NVDIMM module incorporates DRAM, flash, control logic and an independent power source to retain critical in-memory data in the event of unexpected power loss, system crashes or planned shutdowns.

According to Kochak, companies such as Intel, Micron, Sony, Viking Technology, Samsung and Netlist are already shipping NVDIMMs. Moreover, the SNIA (the Storage Networking Industry Association) is working to promote the standard, while a separate group has formed to create a programming model for using persistent memories. This means application developers will ultimately become more of aware of how to fully exploit persistent memory.

“A lot of our software today is driven and developed with one key thing in mind—that DRAM is volatile, you can lose data that is in DRAM, so always design your system in a way that you can expect failures. To do that, there are a lot of things done such as checkpointing, logging and journaling,” he added. “And if you don’t have to do a lot of those operations, you can figure out ways to do things a lot faster and effectively with the same infrastructure and increase the application performance. Then you don’t need to do all of these other operations for dealing with volatile system memory because the system memory would be nonvolatile.”

Interested in learning more about memory in the changing data center? The full text of “Data centers turn to new memories,” by Ann Steffora Mutschler is available on Semiconductor Engineering here, while “Enabling higher system performance with NVDIMM-N” by Sarvagya Kochak can be read on Semiconductor Engineering here.

Rambus’ Aharon Etengoff recently penned an article for Semiconductor Engineering that explores how evolving data center demands are spurring the NVDIMM market. Indeed, a Transparency Market Research (TMR) report published in August confirms that the global Non-Volatile Dual In-line Memory Module (NVDIMM) market is being propelled by an increased demand for advanced data center infrastructure.

NVDIMMs, writes Etengoff, offer fault-tolerant data integrity, while simultaneously optimizing the performance of storage and cache, as well as indexing, message queuing, logging, batch processing, on-line transactions and RAID applications.

“According to TMR, the high performance of the above-mentioned devices is crucial to hyper-scale computing environments that are focused on cloud computing, big data analytics and high-performance database applications,” he explains.

“This is precisely why NVDIMMs are primarily targeted at huge in-memory computing tasks such as ultra-speed in-memory transactional database systems, including those used in search engines and hyper-scale computing applications.”

As Etengoff notes, there are currently three types of JEDEC-compliant NVDIMM implementations: NVDIMM-N, NVDIMM-P and NVDIMM-F. NVDIMM-N, says Etengoff, can perhaps best be described as battery-backed DRAM.

“Essentially, an NVDIMM-N operates just like a standard DRAM DIMM when the CPU is accessing it. However, in the case of a power failure, data is immediately moved from the DRAM to backup NAND on the DIMM itself,” he elaborates. “This ensures the data in memory is never lost. From Rambus’ perspective, NVDIMM-N is a critical stepping stone that will help ensure the success of future SCM and persistent memories on the memory channel.”

Meanwhile, NVDIMM-P allows the host to access both the DRAM and a secondary SCM on the DIMM to enable systems with high-capacity and persistent main memory (further reducing the memory-storage performance and capacity gap), with NVDIMM-F pairing storage DIMMs with traditional DRAM DIMMs.

As Etengoff points out, the NVDIMM-N market recently received a significant boost from Microsoft, with Redmond confirming that Windows 10 Pro for Workstations will support non-volatile memory (NVDIMM-N) hardware, along with ReFS (Resilient File System and faster file sharing). Similarly, Microsoft’s Windows Server 2016 also features support for NVDIMM-N, as does SQL Server 2016. In addition, the Storage Networking Industry Association (SNIA) has announced an update to its non-volatile memory programming model (NPM), which was specifically developed to change how developers think about and interact with main memory.

In conclusion, says Etengoff, the shift from the traditional enterprise data center to the cloud is driving an insatiable demand for increased bandwidth and lower latencies. This has prompted the semiconductor industry to begin the process of redefining conventional memory, storage, network and computing architectures.

“Concurrently, the industry continues to grapple with a waning Moore’s Law. As traditional DRAM scaling slows, there is a collective interest in emerging memories such as NVDIMM which are capable of extending Moore’s Law within the confines of a traditional von Neumann paradigm,” he adds.

Rick Merritt of the EE Times reports that Rambus has working silicon in its labs for DDR5, which the journalist describes as the next major interface for DRAM dual in-line memory modules (DIMMs). The register clock drivers and data buffers, says Merritt, could help double the throughput of main memory in servers, probably starting in 2019.

“To the best of our knowledge, we are the first to have functional DDR5 DIMM chip sets in the lab,” Hemant Dhulla, a vice president of product marketing for Rambus, told the EE Times.

“We are expecting production in 2019 and we want to be first to market to help partners bring up the technology.”

As Merritt notes, DDR5 is expected to support data rates up to 6.4 Gbits/second delivering 51.2 GBytes/s max, up from 3.2 Gbits and 25.6 GBytes/s for today’s DDR4.

“The new version will push the 64-bit link down to 1.1V and burst lengths to 16 bits from 1.2V and 8 bits. In addition, DDR5 lets voltage regulators ride on the memory card rather than the motherboard,” he explained. “In parallel, CPU vendors are expected to expand the number of DDR channels on their processors from 12 to 16. That could drive main memory sizes to 128 Gbytes from 64 GB today.”

DDR5 is expected to first appear on high performance systems running large databases or memory-hungry applications such as machine learning. While some servers may lag adopting DDR5 for six months or so, Dhulla told the EE Times that this scenario will last just a couple of quarters, rather than a couple years.

“Everyone wants a fatter memory pipe… DDR5 is clearly the path to a high-volume opportunity. The big industry debate is what happens beyond DDR5, beyond 2023. [This is why] our labs are looking at multiple alternatives,” he concluded.

As we’ve previously discussed, Rambus recently announced the very first silicon-proven memory buffer chip prototype capable of achieving the speeds required for the upcoming DDR5 standard. Data-intensive applications such as Big Data analytics and machine learning will be key drivers for the adoption of DDR5, with enterprise close behind. Server DIMM chipsets, such as registered clock drivers and data buffers, will be critical to enabling higher memory capacities while maintaining peak performance. Our server DIMM chip prototype leverages the signal integrity and low power, mixed-signal design expertise of Rambus to enable development of next-generation solutions for future data center workloads.

Interested in learning more? You can check out our DDR5 DIMM chipset page here.