John Eble, a senior director of technical product marketing at Rambus, recently penned an article for Semiconductor Engineering about memory buffer chips. As Eble notes, Moore’s Law, which observes that available transistors in an integrated circuit doubles every two years, has driven the semiconductor and IT industries to unparalleled growth over the last 50+ years.
“These transistors have been used in CPUs to increase the number of parallel execution units and instruction fetches, expand the levels of on-chip cache (and overall capacity), support speculative/predictive capabilities, as well as out-of-order and multi-threading hardware,” he explained.
“Moore’s Law has also enabled memory controller support, the expansion of instruction sets for acceleration, the re-architecture of the CPU into a multi-core system and the addition of IO and co-processor interfaces.”
Nevertheless, says Eble, a balanced system is required to realize actual system performance improvements proportional to raw CPU computational increases.
“In 1967, Gene Amdahl theorized that the speedup of parallel computers interconnected to accomplish a workload will be ultimately limited by the portions of the workload that are serial in nature (e.g. data management housekeeping),” Eble continued. “He concluded that the effort in achieving high parallel processing rates will be wasted unless there is an equivalent speedup in the serial portion as well.”
Similarly, says Eble, in today’s multi-core, multi-threaded processors running parallel tasks and parallel virtual servers, the external memory interface bandwidth and capacity are shared resources that can limit the overall speedup achieved unless they also follow the same growth curve as Moore’s Law.
“Memory for servers (as well as for desktops and workstations) are procured and deployed in the form of dual in-line memory modules (DIMM),” he elaborated. “This form factor fits the application space well as it integrates 8-18 DRAM chips in parallel to achieve suitable system bandwidth and minimum capacity, allows for serviceability if a module has ECC errors or goes bad, and allows for expandability through population of previously unused slots or upgrading of existing DIMMs.”
According to Eble, DIMMs have traditionally just been composed of DRAMs, although other memory types such as Flash and 3D Xpoint are now on the horizon.
“On the DIMM itself, multiple ‘ranks’ of DRAM (sets of 8-18 DRAM chips that share the same data, control, and address signal lines as rank 0) can be used to increase capacity of the DIMM,” he explained. “Furthermore, multiple DIMMs can be present on a single CPU memory channel to once again increase overall capacity. With the rise of in-memory compute, database applications, and low latency data analytics, maximizing the amount of server memory per CPU is highly desirable.”
While this topology and form factor are very flexible and allow for a range of memory system capacities, Eble emphasizes that capacity increases come at the cost of loading down the fixed number of channels (composed of data, control, and address signals) per CPU and reduces the achievable bandwidth per channel. However, the addition of memory buffer chips on the DIMM to isolate the full DIMM load from the CPU channel breaks this fundamental tradeoff and enables high system capacity and bandwidth simultaneously.
“This technology was first introduced in standard DIMMs during the SDRAM era of the 90’s and started off as a simple clock repeater. [Clearly], the complexity, capability, and value of memory buffer chips has evolved over the last 20 years and its importance will continue to grow as the industry tries to sustain Moore’s Law and the requisite scaling of memory bandwidth and capacity,” he concluded.
