Samsung’s solution runs on theCompute Express Link (CXL), an open industry standard that provides a cache-coherent interconnect between CPUs and accelerators, thus allowing CPUs to use the same memory regions as connected devices utilizing CXL. The remote memory, or in this case, a hybrid RAM/flash memory device, is accessible over the PCIe bus, which comes at the cost of ~170-250ns of latency, or roughly the cost of a NUMA hop. CXL was introduced in 2019 and is in its third revision, featuring PCIe 6.0 support.
The CXL spec supports three types of devices: Type 1 devices are accelerators that lack local memory, Type 2 devices are accelerators with their own memory (like GPUs, FPGAs, and ASICs with DDR or HBM), and Type 3 devices consist of memory devices. The Samsung device falls into the Type 3 category.
CMM-H TM is an offshoot of Samsung’s CMM-H CXL memory solution. Samsung says it is the world’s firstFPGA-based tiered CXL memory solution and is designed to “tackle memory management challenges, reduce downtime, optimize scheduling for tiered memory, and maximize performance, all while significantly reducing the total cost of ownership.”
This new CMM-H isn’t as fast as DRAM; however, it adds a beefy slab of capacity via the flash but hides a lot of latency with a clever memory caching feature built into the expansion card. Hot data is moved to the card’s DRAM chips to improve speed, while less used data is stored in NAND storage. Samsung says this behavior happens automatically, but some applications and workloads can give the device hints to improve performance through an API. Naturally, this will add some latency for cached data, which isn’t ideal for all use cases, particularly those that rely on tight 99th percentile performance.
Samsung’s new expansion card will provide its customers with new ways to expand their server’s memory capacity. This new design paradigm is becoming more important as more advanced large language models continue to demand more memory from their host machines and accelerators.
Get Tom’s Hardware’s best news and in-depth reviews, straight to your inbox.
Aaron Klotz is a contributing writer for Tom’s Hardware, covering news related to computer hardware such as CPUs, and graphics cards.
