Server DIMM Chipsets

Earlier this month, Semiconductor Engineering’s Ann Steffora Mutschler penned an article that takes a closer look at how buffering is gaining ground as a way to speed up the processing of increasingly large quantities of data. In simple terms, says Mutschler, a data buffer is an area of physical memory storage that temporarily stores data while it is being moved from one place to another.

“This becomes increasingly necessary in data centers, autonomous vehicles and for machine learning applications,” she explains. “The challenges with these applications are advanced signal equalization and increased capacity and bandwidth. Data buffering techniques — either as a discrete chip within the memory module or integrated into an SoC — help make this all possible.”

As Mutschler notes, buffer chips manage read and write commands from the CPU to the DRAM. When buffer chips are used, it is known as a load reduced dual inline memory module (LRDIMM). Those chips boost total memory capacity at the same performance and speed.

“As data is growing at exponential rates, reducing CPU load through data buffer chips becomes necessary to increase DRAM capacity per CPU,” Victor Cai, director of product marketing for Rambus‘ Memory and Interfaces Division, told the publication. “LRDIMMs are common in data centers that are performing data-intensive applications, such as big data analytics, artificial intelligence and machine learning.”

As Cai noted in a previous Semiconductor Engineering article, DDR3 server DIMM chipsets (800 Mbps) first hit the market in 2006 and began to ramp the following year. By the time DDR4 server DIMM chipsets (2133) began shipping in 2014, DDR3 server DIMM chipsets were spanning the following five speeds: 800, 1066, 1333, 1600 and 1866. In the last years, DDR4 buffer chipset shipments have crossed over in term of volume, with DDR4 chipset speeds expected to reach 3200 by 2018 or 2019.

In addition to increased bandwidth and density, DDR4 buffer chips offer data centers the critical benefits of advanced signal equalization. For example, Rambus registered clock driver (RCD) and data buffer (DB) chips are equipped with sophisticated decision feedback equalizers (DFEs) which are typically used in the communication industry for 10G or higher SerDes. Essentially, DFE circuits allow RCDs and DBs to operate with greater margin on typical systems – as well as in challenging environments where older chipsets are prone to malfunction or fail to achieve higher speeds.

Equipping both chips (RCD and DB) with DFE capabilities allows the DIMM chipset to push the limits of both the command address bus as well as the DQ bus. Moreover, DFE circuits enable DIMMs to run at speeds of 3200, exceeding original industry expectations for this generation of buffer chips. It is also important to emphasize that signal integrity challenges are becoming more prevalent as bandwidth, density and speeds increase. Fortunately, lessons learned from other markets (such as communications) will merge with DDR technology to help push the single-ended bus to its limit.

From our perspective, memory buffer chips will continue to steadily evolve, as is illustrated by the number of technical advances (such as DFE) seen from one generation of silicon to the next. The importance of server DIMM chipsets will only continue to increase, as the industry attempts to sustain Moore’s Law and maximize the limitations of Von Neumann architecture in a changing data center.

Interested in learning more about data buffering? The full text of “Data Buffering’s Role Grows” by Ann Steffora Mutschler can be read on Semiconductor Engineering here.

Victor Cai, a director of product marketing at Rambus, recently wrote an article for Semiconductor Engineering that explores the evolution of server DIMM chipsets in the data center.

According to Cai, DDR3 server DIMM chipsets running at 800 Mbps first hit the market in 2006 and started to ramp in 2007. 7 years later, DDR4 server DIMM chipsets running at 2133 began shipping. Concurrently, DDR3 server DIMM chipsets spanned the following five speeds in 2014: 800, 1066, 1333, 1600 and 1866.

“In the last years, DDR4 buffer chipset shipments have crossed over in term of volume, with DDR4 chipset speeds expected to reach 3200 by 2018 or 2019,” Cai explains. “In addition to increased bandwidth and density, DDR4 buffer chips offer data centers the critical benefits of advanced signal equalization.”

As an example, says Cai, Rambus registered clock driver (RCD) and data buffer (DB) chips are equipped with sophisticated decision feedback equalizers (DFEs) which are typically used in the communication industry for 10G or higher SerDes.

“Essentially, DFE circuits allow RCDs and DBs to operate with greater margin on typical systems – as well as in challenging environments where older chipsets are prone to malfunction or fail to achieve higher speeds,” states Cai. “Equipping both chips (RCD and DB) with DFE capabilities allows the DIMM chipset to push the limits of both the command address bus as well as the DQ bus.”

Moreover, says Cai, DFE circuits enable DIMMs to run at speeds of 3200, exceeding original industry expectations for this generation of buffer chips. However, signal integrity challenges are becoming more prevalent as bandwidth, density and speeds increase.

“Fortunately, lessons learned from other markets (such as communications) will merge with DDR technology to help push the single-ended bus to its limit,” he notes.

With regards to current DIMM chipset deployment, Cai observes that DDR4 buffer chipset shipments are ramping up as the price of DDR4 silicon continues to decrease. This, says Cai, is because data centers only upgrade their memory subsystems when the economics of performance per dollar meets a certain threshold.

“For most data centers, this means a transition from DDR3 to DDR4 within the DDR4-2400 generation – as price parity is reached between DDR3 DRAM and DDR4 DRAM,” he elaborates. “Many industry heavyweights have already transitioned or are in the process of switching over to DDR4 as their primary server memory type. Looking towards the near future, speed increases and price reductions will further accelerate adoption of DDR4 server DIMM chipsets by data center customers.”

In conclusion, says Cai, memory buffer chips continue to steadily evolve, as is illustrated by the number of technical advances (such as DFE) seen from one generation of silicon to the next.

“The importance of server DIMM chipsets will only continue to increase, as the industry attempts to sustain Moore’s Law and maximize the limitations of Von Neumann architecture in a changing data center,” he adds.

DDR5

Ann Steffora Mutschler recent penned an article for Semiconductor Engineering that explores the role of new memory technology in the evolving data center. For example, DDR5 is expected to offer significant improvements over previous iterations of DRAM, including doubling the bandwidth and density over DDR4, as well as facilitating optimized channel efficiency. Nevertheless, DDR5 cannot stand alone as the industry struggles to meet the tsunami of data generated by an increasingly connected world.

“The amount of data being generated by sensors in a variety of connected devices across multiple markets is exploding and so is the demand for compute power to make sense of [it] all,” Mutschler explained. “Machine learning/AI/deep learning, separately and as extensions of markets filled with connected devices, are only adding to the need for additional types of memories.”

DRAM scaling slows

In addition to DDR5, the industry is eyeing GDDR5 (originally targeted at the graphics market), HBM2 and NVDIMM (persistent memory). This is because no single memory type is capable of satisfying the insatiable demand for ever more bandwidth and capacity, especially as Moore’s Law wanes and DRAM scaling slows.

“It’s been difficult for large memory manufacturers like Samsung, SK Hynix and Micron to get the densities higher and higher. Right now, 4- and 8-gigabit devices are shipping in mass production in high volumes,” Sarvagya Kochak, a senior product marketing manager at Rambus, told Semiconductor Engineering. “16 gigabit should be on the market in a year or two, but this poses a pretty significant challenge because if you look at it from a CPU perspective, CPU speeds haven’t gone up over the last decade. They have more or less leveled off because transistor speeds cannot be bumped up more than they are right now.”

To compensate, says Kochak, CPU vendors are adding more cores inside of processors, although this presents its own set of challenges.

“The problem with this approach is that within the CPU is an integrated memory controller, which interfaces with the DRAM DIMMs. With older CPUs, there would have been two memory controllers. Now there are four memory controllers. Soon there will be six memory controllers and the projection is that there may even be eight memory controllers per CPU,” he elaborated. “The reason why there are more memory controllers is because there needs to be more data to feed all of the cores, so if there are fewer memory controllers, the data can’t be fed from the DRAM to the cores. If the cores are hungry, there’s no point in having them. As the number of cores goes up, there is a requirement for more bandwidth from the DRAM into the system. If you look at the rate at which memory has been increasing, as well as the memory attach rate per CPU, it’s not a pretty picture.”

NVDIMMs (persistent memory)

As Kochcak previously explained in a detailed Semiconductor Engineering article, the above-mentioned DRAM challenges have prompted system and software architects to rethink standard memory-storage subsystems in an effort to extract all the performance they possibly can with currently available technologies. More specifically, the majority of application, OS and software stacks are (presently) designed with the underlying assumption that memory is volatile and the contents in DRAM can be lost. Complex mechanisms ensure little to no data is lost in the event of a power failure. Although this comes at the cost of performance, the industry has learned to accept the tradeoff.

With non-volatile DIMMs (NVDIMM) technology, it is possible to accelerate system performance by making memory persistent and eliminating the aforementioned tradeoff. As Mutschler observes, NVDIMMs operate as standard DRAM, while also offering the persistence of flash. A typical NVDIMM module incorporates DRAM, flash, control logic and an independent power source to retain critical in-memory data in the event of unexpected power loss, system crashes or planned shutdowns.

According to Kochak, companies such as Intel, Micron, Sony, Viking Technology, Samsung and Netlist are already shipping NVDIMMs. Moreover, the SNIA (the Storage Networking Industry Association) is working to promote the standard, while a separate group has formed to create a programming model for using persistent memories. This means application developers will ultimately become more of aware of how to fully exploit persistent memory.

“A lot of our software today is driven and developed with one key thing in mind—that DRAM is volatile, you can lose data that is in DRAM, so always design your system in a way that you can expect failures. To do that, there are a lot of things done such as checkpointing, logging and journaling,” he added. “And if you don’t have to do a lot of those operations, you can figure out ways to do things a lot faster and effectively with the same infrastructure and increase the application performance. Then you don’t need to do all of these other operations for dealing with volatile system memory because the system memory would be nonvolatile.”

Interested in learning more about memory in the changing data center? The full text of “Data centers turn to new memories,” by Ann Steffora Mutschler is available on Semiconductor Engineering here, while “Enabling higher system performance with NVDIMM-N” by Sarvagya Kochak can be read on Semiconductor Engineering here.

Rambus’ Aharon Etengoff recently penned an article for Semiconductor Engineering that explores how evolving data center demands are spurring the NVDIMM market. Indeed, a Transparency Market Research (TMR) report published in August confirms that the global Non-Volatile Dual In-line Memory Module (NVDIMM) market is being propelled by an increased demand for advanced data center infrastructure.

NVDIMMs, writes Etengoff, offer fault-tolerant data integrity, while simultaneously optimizing the performance of storage and cache, as well as indexing, message queuing, logging, batch processing, on-line transactions and RAID applications.

“According to TMR, the high performance of the above-mentioned devices is crucial to hyper-scale computing environments that are focused on cloud computing, big data analytics and high-performance database applications,” he explains.

“This is precisely why NVDIMMs are primarily targeted at huge in-memory computing tasks such as ultra-speed in-memory transactional database systems, including those used in search engines and hyper-scale computing applications.”

As Etengoff notes, there are currently three types of JEDEC-compliant NVDIMM implementations: NVDIMM-N, NVDIMM-P and NVDIMM-F. NVDIMM-N, says Etengoff, can perhaps best be described as battery-backed DRAM.

“Essentially, an NVDIMM-N operates just like a standard DRAM DIMM when the CPU is accessing it. However, in the case of a power failure, data is immediately moved from the DRAM to backup NAND on the DIMM itself,” he elaborates. “This ensures the data in memory is never lost. From Rambus’ perspective, NVDIMM-N is a critical stepping stone that will help ensure the success of future SCM and persistent memories on the memory channel.”

Meanwhile, NVDIMM-P allows the host to access both the DRAM and a secondary SCM on the DIMM to enable systems with high-capacity and persistent main memory (further reducing the memory-storage performance and capacity gap), with NVDIMM-F pairing storage DIMMs with traditional DRAM DIMMs.

As Etengoff points out, the NVDIMM-N market recently received a significant boost from Microsoft, with Redmond confirming that Windows 10 Pro for Workstations will support non-volatile memory (NVDIMM-N) hardware, along with ReFS (Resilient File System and faster file sharing). Similarly, Microsoft’s Windows Server 2016 also features support for NVDIMM-N, as does SQL Server 2016. In addition, the Storage Networking Industry Association (SNIA) has announced an update to its non-volatile memory programming model (NPM), which was specifically developed to change how developers think about and interact with main memory.

In conclusion, says Etengoff, the shift from the traditional enterprise data center to the cloud is driving an insatiable demand for increased bandwidth and lower latencies. This has prompted the semiconductor industry to begin the process of redefining conventional memory, storage, network and computing architectures.

“Concurrently, the industry continues to grapple with a waning Moore’s Law. As traditional DRAM scaling slows, there is a collective interest in emerging memories such as NVDIMM which are capable of extending Moore’s Law within the confines of a traditional von Neumann paradigm,” he adds.

Rick Merritt of the EE Times reports that Rambus has working silicon in its labs for DDR5, which the journalist describes as the next major interface for DRAM dual in-line memory modules (DIMMs). The register clock drivers and data buffers, says Merritt, could help double the throughput of main memory in servers, probably starting in 2019.

“To the best of our knowledge, we are the first to have functional DDR5 DIMM chip sets in the lab,” Hemant Dhulla, a vice president of product marketing for Rambus, told the EE Times.

“We are expecting production in 2019 and we want to be first to market to help partners bring up the technology.”

As Merritt notes, DDR5 is expected to support data rates up to 6.4 Gbits/second delivering 51.2 GBytes/s max, up from 3.2 Gbits and 25.6 GBytes/s for today’s DDR4.

“The new version will push the 64-bit link down to 1.1V and burst lengths to 16 bits from 1.2V and 8 bits. In addition, DDR5 lets voltage regulators ride on the memory card rather than the motherboard,” he explained. “In parallel, CPU vendors are expected to expand the number of DDR channels on their processors from 12 to 16. That could drive main memory sizes to 128 Gbytes from 64 GB today.”

DDR5 is expected to first appear on high performance systems running large databases or memory-hungry applications such as machine learning. While some servers may lag adopting DDR5 for six months or so, Dhulla told the EE Times that this scenario will last just a couple of quarters, rather than a couple years.

“Everyone wants a fatter memory pipe… DDR5 is clearly the path to a high-volume opportunity. The big industry debate is what happens beyond DDR5, beyond 2023. [This is why] our labs are looking at multiple alternatives,” he concluded.

As we’ve previously discussed, Rambus recently announced the very first silicon-proven memory buffer chip prototype capable of achieving the speeds required for the upcoming DDR5 standard. Data-intensive applications such as Big Data analytics and machine learning will be key drivers for the adoption of DDR5, with enterprise close behind. Server DIMM chipsets, such as registered clock drivers and data buffers, will be critical to enabling higher memory capacities while maintaining peak performance. Our server DIMM chip prototype leverages the signal integrity and low power, mixed-signal design expertise of Rambus to enable development of next-generation solutions for future data center workloads.

Interested in learning more? You can check out our DDR5 DIMM chipset page here.