Why Latency Impacts SSD Performance More Than Bandwidth Does

One of the best and most cost-effective ways to improve the performance of an old PC is to put an SSD in it. Even a purpose-built legacy system running the highest-end components available in its era can be further improved by an SSD. Systems that don’t support the old SATA standard can often add SATA ports via the old 32-bit PCI bus.

SSDs are arguably unique in this regard. It is impossible to retrofit a modern PCIe GPU into an Intel 440BX motherboard for Win 9x gaming or to drop a stick of 16GB DDR4-3200 into a Pentium 4, but you can buy a modern SATA SSD and a PCI SATA card and build yourself a Windows 2000 or Windows XP system that blows any HDD-equipped system of its age out of the water. Even if it was possible to rig a modern GPU in a legacy system this way, the end result would be underwhelming as the CPU failed to keep the GPU fed.

One niche use case for the old 3dfx Voodoo 5 5500 was to pair it with a weaker CPU and devote the second GPU core to anti-aliasing. If your CPU wasn’t fast enough to drive the entire card — and this was the case for some lower-end chips — you could get arguably “free” antialiasing relative to the maximum performance your system was capable of. While a single CPU core won’t push a modern SATA SSD very hard either, the relationship between CPU and HDD performance is quite different than the relationship between CPU and GPU performance.

Over at ZDNet, Robin Harris discusses why latency is so much more important than bandwidth in SSD performance. While he doesn’t discuss legacy computing, his answer explains why even old machines benefit so much from a storage upgrade.

Disk manufacturers like to quote bandwidth because it’s easy to measure and understand. As Harris points out, however, bandwidth doesn’t measure how long it takes the computer to service a request. A hard drive with infinite bandwidth and a seek time of 10ms can respond to a maximum of 100 data accesses per second, whether those requests are for 1MB or 1TB. If there are 10,000 I/O requests backed up, the end-user is going to notice the lag. Because typical I/O operations don’t consume much bandwidth, how quickly a system can respond to multiple storage access requests is typically more important than the amount of bandwidth they can dedicate to any single task.

Access latency is where SSDs shine. SATA SSDs have an access latency of ~70 microseconds according to this WD blog, compared with ~10-15ms for a typical HDD. Figures quoted for SATA SSDs vary widely, with values generally clustered between 100-200 microseconds. Call it 12.5ms for the hard drive, and the SATA SSD is between 62x -125x faster at responding to service requests than the hard drive it replaces.

A SATA SSD in an old system might not support more than SATA I transfer rates of ~120-130MB/s (accounting for overhead). Hard drives of the era, like the WD800JB, were capable of sustaining 49MB/s outer transfer rates. The ~3x bandwidth improvement is large, but not nearly as large as the enormous improvement in storage latency. Assume an old chip gets one-third the benefit of a modern one, and we’re still talking about a 20x improvement over native HDD. These latency improvements translate directly into faster performance, even on legacy hardware.

The reason even an ancient system can get such a kick from an SSD upgrade is that hard drives were an anchor to absolute PC speed for a very long time. The situation is worse now — HDD performance has not improved nearly as much as CPU performance over the past two decades — but it was a meaningful drag on systems even in the 1990s. This is not to imply that bandwidth can’t be a limiting factor to system performance, but it’s not the primary reason why SSDs feel so much faster than hard drives even for ordinary desktop use.

This is also why it’s much harder to see the difference between an NVMe drive and a SATA SSD, compared with the gap between a SATA SSD and a hard drive. Intel claims Optane can service storage requests in as little as 14 microseconds. Even if we assume the 200 microsecond SATA SSD latency is accurate, the performance improvement from SATA to NVMe-attached Optane is “only” 14.3x.

We’ve long noted that most consumer applications are much more latency-sensitive than bandwidth-sensitive. But discussing the issue in terms of maximum addressable storage requests per second is an interesting way to look at the problem and a good explanation for why SSDs are so helpful across such a long span of time. A modern GPU paired with an old CPU would not deliver a 20-60x performance improvement compared with legacy hardware. A modern SSD compared against a legacy hard drive just might.

Continue reading

Nvidia Unveils Ampere A100 80GB GPU With 2TB/s of Memory Bandwidth

Nvidia announced an 80GB Ampere A100 GPU this week, for AI software developers who really need some room to stretch their legs.

JEDEC Extends HBM2 Standard to 24GB, 307GB/s Bandwidth Per Stack

JEDEC has updated the HBM2 standard for higher transfer speeds and increased density.

New Intel Compute Express Link Boosts Accelerator, CPU Bandwidth

Intel and a group of additional technology companies have announced a new common interconnect standard, CXL. The new standard should be ready to debut in hardware by 2021.

Google Stadia Will Eat 1TB Bandwidth Caps for Breakfast

Google's Stadia service comes with bandwidth consumption requirements that could put a serious strain on 1TB data caps across the country. 4K isn't going to be accessible to a lot of folks — not without overage fees.