Loren Shalinsky, a Strategic Development Director at Rambus, recently penned a detailed article for Semiconductor Engineering that explores the memory-storage hierarchy.
As he puts it, the hierarchy, or pyramid, is a particularly succinct method of understanding computer systems and the dizzying array of memory options available to the system designer.
“Many different parameters characterize the memory solution,” Shalinsky explained. “Among them are latency (how long the CPU needs to wait before the first data is available) and bandwidth (how fast additional data can be ‘streamed’ after the first data point has arrived), although by my count there are more than 10 different parameters to measure.”
As expected, no single memory sub-system can be considered “best” in all categories. As such, various memory solutions are routinely exploited at different levels of the hierarchy to achieve optimized results. For example, high-end systems, such as servers found in datacenters, are most likely to leverage solutions from every level in the hierarchy.
While not changing the relative placement on the pyramid (see above), memory systems continue to evolve at a steady cadence. As such, future DIMM subsystem improvements are perhaps the easiest to imagine. To be sure, DRAM latency has not changed much over the years, although DRAM data rates continue to increase with an eye on more capacity and bandwidth.
New memory technologies such as HBM or HMC, says Shalinsky, can be sandwiched in-between DIMMs and on-chip memories – with the ability to place gigabytes of data even closer to the CPU than a DIMM.
“Going back 5-10 years, Solid State Drives (SSDs) started to fill the huge gap that originally existed between DIMMs and hard drives,” he continued. “[However], the underlying NAND technology performance has somewhat leveled off (but made extraordinary progress in price reduction), and has therefore left the door open for additional technologies to fill the remaining gaps.”
To be sure, 3D XPoint technology, announced by Intel and Micron earlier this month, seems to be targeting these very gaps.
“While technical details are scarce, we can piece together enough data points to surmise that 3D XPoint could fill one of the two blank levels currently in between SSDs and DIMMS,” he added. “Even with the addition of 3D XPoint, many gaps will continue to exist in the memory hierarchy, leaving no shortage of research avenues for companies in the memory industry.”
It should be noted that Shekhar Borkar, Intel Fellow and director of extreme-scale technologies, recently told The Platform DRAM will be regarded as a first-level, high capacity memory for years to come.
“The bottom line is that for the next ten years, if I am a node designer, I will rely on DRAM as a first-level, high capacity memory, followed by NAND or PCM as the next level for storage,” he said. “Everything else – keep working on it, and when it is ready, I will use it. Today, you are not ready.”