Rest in Peace, Optane

Intel’s Optane memory modules launched with a lot of fanfare in 2015, and were recently
discontinued, in 2022, with similar fanfare. It was a sad day for me, a lover of
abstraction-breaking technologies, but it was forseeable and understandable.

At the time of Optane’s launch, a lot of us were excited about the idea of having a new storage
tier, sitting between DRAM and flash. It was announced as having DRAM endurance and speed with
the persistence and size of flash. It was a futuristic memory technology, but the technology of
the future met the full force of Wright’s Law.

Wright’s Law

Each doubling of production volume corresponds to a 20% decrease in cost.

This is the simple statement of Wright’s Law. It was originally developed by a man named Theodore
Wright in the 1930’s when looking at the production of airplane parts. Since then, it has been
verified to hold in many different industries, although the cost reduction seems to vary between
10 and 25% depending on the industry. Despite the industry difference, the history of
semiconductor technology has actually followed Wright’s Law more closely than Moore’s Law: The
density increases predicted by Moore’s Law tend to speed up and slow down with demand growth,
exactly as predicted by Wright’s Law.

During the last 7 years, DRAM and Flash have both experienced massive increases in density and
production volume, and in turn, Wright’s Law has given us cheaper, faster, bigger devices.

DRAM Production 2015-2022

The lifetime of DDR4 memory roughly matches the lifetime of Optane. DDR4 was introduced a few
years before 2015, and became cost-effective compared to DDR3 around 2015-2016. Today, DDR5 RAM
and the CPUs that can use it have just recently been introduced to the market. Over this time,
RAM price per bit has roughly halved, and global RAM production has grown by a factor of about
10-15. However, due to competition between AMD and Intel, the number of memory channels per CPU
has also roughly doubled. Thus, a server in 2022 holds about 4x as much RAM as a server in 2015,
and memory cost as a share of system cost has increased by a factor of 2. Memory bandwidth has
increased by a factor of about 3 due to a doubling of memory channels and the channels getting
faster.

This means that both the capacity argument of Optane (“a system with Optane DIMMs can hold more
memory”) has diminished over time, and the performance gap between Optane and DDR memory has also
grown. A two-socket server populated with RDIMMs can now hold several TB of memory, so the value
proposition of Optane for memory capacity has been eliminated.

Flash is a Spectrum

Flash memory comes in four flavors, based on the number of bits stored in each cell. Single-level
cell (SLC) flash stores one bit per cell. It has the lowest density, but the longest endurance and
highest speeds. MLC (multi-level cell) holds two bits per cell, and is worse on speed and
endurance than SLC, but better than TLC (triple-level cell). Most SSDs today use TLC flash: it
offers a good balance between price and endurance, and most workloads are read-heavy, so there
isn’t a lot of stress on the memory arrays. QLC (quad-level cell) has recently been introduced to
the market as a capacity SSD technology: QLC has even lower endurance and speed than TLC, but
increases the capacity a lot. PLC (penta-level) is on the horizon, promising to continue the
trend.

On the back of this distinction, we can see a natural tiering of flash: using TLC or QLC for
capacity (in place of disk for all but the largest datasets), and using SLC for caching and
write-heavy workloads. This means that SLC flash is a direct competitor to Optane, but it
has the advantages of being much cheaper per bit without being a lot slower.

Flash Production 2015-2022

After Optane’s launch, Flash memory technology advanced quickly. In 2015, NVMe drives had been out
for about a year or two. NVMe drives and the controller chips that ran them were still working out
the kinks. Understandably, Intel also created NVMe drives with Optane, offering much better read
and write latencies than NVMe drives with flash. Not only did Optane offer faster access than
flash, but it had the advantage of being a simpler, more reliable form of memory that didn’t need
complicated controller algorithms.

As time went on, companies quicly improved the speed, reliability, and density of each flash
technology. In 2015, TLC was roughly in the position QLC occupies today: mostly for read-only and
read-heavy workloads. Now, TLC is the workhorse flash technology used for general-purpose drives,
as its speed and endurance have improved. Similarly, SSD controller chips have also improved over
time, allowing them to work faster and make better use of the flash chips on an SSD. Lots of money
goes into R&D work for flash, and it shows.

In the last 7 years, flash has grown to be the dominant storage technology: going from almost 20x
more per bit than hard drives to just 5x more per bit. In addition, SSD latency has dropped by
a factor of 2 due to control chips getting more powerful and algorithms getting better. Flash
bandwidth has increased to the point where an SSD can saturate a PCIe gen 4 x4 link. A modern
SLC SSD can be found with read latency under 30 microseconds, and even TLC SSDs can get close to
100 microsecond latencies.

Flash has advanced so much over the last 7 years that if the trend continues, I may be eulogizing
the magnetic hard drive in 2030. Flash is eating the storage world for good reason: it can offer
high speeds and high capacity, and while it had some rough edges, we have learned how to work with
it very well.

Optane and Wright’s Law

Clearly, as time has advanced, Optane’s position in the memory hierarchy has been getting crushed
between expanding DDR4 DRAM and flash that is getting faster and faster. Both of these
technologies have had much more innovation and R&D over the last 7 years than Optane, and it shows.
In retrospect, Intel was fighting an uphill battle if they wanted to treat Optane simply as a point
on the memory hierarchy. To make the project successful, they needed it to be a unique value add.

The Value Proposition of Optane

After the dust settled, Optane modules only ended up having a small price difference with DRAM
modules of the same size, but were almost as slow as SLC flash, and only had 3-5x the endurance of
flash memory. The NVMe drives were faster, but not that much faster, than SLC flash. The only
value-add that was left was the abstraction difference with normal memory: Optane offered
persistence on the memory bus.

If you have worked on a database or a storage system, you will know how valuable this can be: lots
of code in high-throughput storage systems is there to make sure that transactions persist as
quickly as possible. Conceivably, if the normal memory writes involved in that transaction can
make it persistent, you can delete a ton of code. You could even conceivably run a small database
on an mmap-ed Optane file, and persistence would be given to you for free. Oh wait… a lot of
databases mmap their backing files already.

Persistence on the Memory Abstraction

It turns out that what developers mostly would have wanted from Optane they got from BSD and Linux
in the form of mmap. mmap-ing a file allows you to treat the contents of the file as though
they are in memory, and allows the filesystem and a cache to handle the rest. The parts of the
file you have accessed recently are in memory, while other parts of the file are fetched from disk
when you demand them – your access triggers a page fault, which in turn triggers a read from the
filesystem. Writes to the file are handled in the background. This is not the fastest way to
access files, but it is a very convenient one, and it works well enough for many high-performance
key-value databases like LevelDB, LMDB, SQLite, QuestDB, RavenDB, and more.

mmap had one more secret weapon: the filesystem. If the media backing the filesystem wore out,
the filesystem (and the layers underneath it) could detect and correct issues. Optane had no
such protection.

When Intel thought they were enabling a whole new class of high-performance databases by offering
persistence on the memory bus, they were really offering a performance boost compared to persistent
databases using mmap. It might not even have been a performance boost: Optane slowed down your
median read and write operations in exchange for avoiding those page faults, so if your working set
was small or mostly fit inside RAM, mmap and NVMe flash was actually faster.

Persistent Memory and Caches

It turns out that offering persistence on the memory bus runs into abstraction problems with the
current memory controllers on CPUs. The memory controllers and cache hierarchies are designed
assuming that memory is dumb and volatile, and they can indefinitely delay writes to save
bandwidth.

The initial solution that Intel came up with was an instruction, CLFLUSH, that flushed a cache
line to memory. However, CLFLUSH was a serializing instruction, like CPUID, so it ended up
flushing the CPU pipeline as well as the cache hierarchy when it was issued. Worse, flusing the
cache line would invalidate it, so if you wanted to read the value back after writing it, you would
incur a cache miss. It was later supplemented by CLFLUSHOPT and CLWB which also could be used
to flush and write back cache lines without incurring the same performance penalty.

However, when Optane memory started to come out, CLFLUSH was the only instruction from this suite
that was available, meaning that initial performance tests suffered from both the speed difference
between Optane and DRAM and the substantial overhead of the CLFLUSH instruction. Intel would
probably have had a much easier time selling Optane if their core architecture were ready.

Alternative Methods of Persistence

Also between 2015 and 2022, several companies were offering a different kind of persistent DIMM.
Instead of using a special type of memory, this kind of persistent DIMM used normal DRAM, but added
some flash memory and a small controller circuit on the back that saved the contents of the DRAM
to flash every time the power started to dip.

To make sure that the contents of the DIMM were safe, these persistent DIMMs had a supercapacitor
or small lithium ion battery (usually kept in a 2.5 inch drive bay and connected to the DIMM
through a cable) that kept the DIMM powered while the rest of the system was going down. On
power-up, the DIMM would restore the memory contents.

These were later specified by JEDEC as “NVDIMM-N” modules (standing for Non-volatile Dual Inline
Memory Module – NAND flash type).

Still, these alternatives have some problems: they are a lot thicker than normal memory DIMMs, and
they can’t offer the capacity that Optane modules could offer. However, they don’t have the same
durability problems that Optane and flash do, since the flash is rarely written, and they operate
at the same speed as the rest of the system’s memory.

Thank You, Optane

So many technologies have become successful around Optane and the promises it held. Unfortunately,
Optane was not one of them.

SSDs using SLC flash offer blazing fast performance with a block abstraction, and enterprises and
database developers learned how to take advantage of the differences between SSDs that were small
and fast, using SLC flash, and SSDs that were slower and larger, with TLC and QLC flash. Some SSD
manufacturers also saw the idea of a caching drive and competed by adding SLC caching to their TLC
SSDs. Most high-end consumer SSDs today do this for you.

For the few customers who needed persistence on the memory bus, the JEDEC NVDIMM standards emerged,
with the flagship NVDIMM-N modules allowing you to have a DRAM module that was both persistent
and fast, and an additional standard to cover future persistent memory technologies. Intel’s
new instructions allow users to take advantage of these new modules, adding a fundamental
capability to CPUs.

Optane has helped us learn to build computing systems that take advantage of the spectrum of
“legacy” storage and memory technologies. I, for one, am sad to see it go, but happy that I won’t
miss it.