Maximizing Hybrid Memory Systems with NOVA
NOVA – a file system designed to maximize performance on hybrid memory systems – was introduced in 2016 by Jian Xu and Steven Swanson of the University of California, San Diego. Essentially, NOVA adapts conventional log-structured file system techniques to exploit the fast, random access that NVMs deliver. NOVA maintains separate logs for each inode to improve concurrency and stores file data outside the log to minimize size and reduce garbage collection costs. Moreover, NOVA’s logs provide metadata, data and mmap atomicity. In short, NOVA focuses on simplicity and reliability by keeping complex metadata structures in DRAM to accelerate lookup operations.
NOVA and Intel’s Optane DC Persistent Memory Module
NOVA was recently cited in a paper published by the Non-Volatile Systems Laboratory, which operates under the auspices of the Computer Science & Engineering University of California, San Diego. The paper, penned by Joseph Izraelevitz and additional researchers (including Steven Swanson), explores the basic performance measurements of Intel’s Optane DC Persistent Memory Module.
According to the paper, Intel’s nonvolatile DIMM supports byte-granularity accesses with access times on the order of DRAM, while also providing data storage that survives power outages. As the researchers note, Optane DC memory occupies a tier in-between SSDs and DRAM. It boasts higher latency (346 ns) than DRAM, although lower latency than an SSD. As opposed to DRAM, its bandwidth is asymmetric with respect to access type. Max read bandwidth is 33.2 GB/s and scales with thread count, while its maximum write bandwidth is 8.9 GB/s and peaks at only four threads.
“Optane DC PMMs can be used as large memory devices with a DRAM cache to hide their lower bandwidth and higher latency. When used in this Memory (or cached) mode, Optane DC memory has little impact on applications with small memory footprints. Applications with larger memory footprints may experience some slow-down relative to DRAM, [although they] are now able to keep much more data in memory,” the researchers explain.
Image Credit: Non-Volatile Systems Laboratory, the Computer Science & Engineering University of California, San Diego (https://arxiv.org/pdf/1903.05714.pdf)
“In contrast, in App Direct (or uncached) mode, Optane DC PMMs can be used as a persistent storage device. When used under a file system, this configuration can result in significant performance gains, especially when the file system is optimized to use the load/store interface of the Optane DC PMM and the application uses many small, persistent writes. For instance, using the NOVA-relaxed NVMM file system, we can improve the performance of Kyoto Cabinet by almost 2×.”
Raising the memory Ziggurat
The NOVA NVM-based file system also serves as the basis for an NVM optimization tool known as Ziggurat. Essentially, Ziggurat exploits the benefits of NVM through intelligent data placement during file writes and data migration. More specifically, Ziggurat includes two placement predictors that analyze the file write sequences and predicts whether the incoming writes are both large and stable and whether updates to the file are likely to be synchronous.
Image Credit: University of California, San Diego (https://arxiv.org/pdf/1904.11560.pdf)
Subsequently, Ziggurat routes the incoming writes to the most suitable tier based on the prediction – with writes to synchronously-updated files going to the NVM tier to minimize the synchronization overhead. Small, random writes also go to the NVM tier to entirely avoid random writes to disk. The remaining large sequential writes to asynchronously-updated files go to disk.
According to Morteza Hoseinzadeh of the University of California, San Diego, Ziggurat seeks five primary design goals: Send writes to the most suitable tier, only migrate cold data in cold files, high NVM space utilization, migrate file data in groups and high scalability. In a nutshell, says Hoseinzadeh, Ziggurat bridges the gap between disk-based storage and NVM-based storage and provides high performance and large capacity to applications.
The Future of SCMs
Commenting on NOVA and Ziggurat, Steven Woo, Rambus fellow and distinguished inventor, tells Rambus Press that the work of Hoseinzadeh and Izraelevitz shows that the industry’s efforts to bring storage class memories (SCMs) to market have a lot of potential.
“Tiering is a logical way to integrate SCMs into memory and storage hierarchies. However, the newness of the technology means that the industry must continue doing research to determine how best to obtain the advantages that these memories have to offer. Both Ziggurat and NOVA illustrate the importance of re-thinking assumptions about file systems and the memory/storage hierarchy in order to get the most out of SCMs,” he concludes.