ARM’s First Client PC Roadmap Makes Bold Claims, Doesn’t Back Them Up

ARM’s First Client PC Roadmap Makes Bold Claims, Doesn’t Back Them Up

ARM is announcing its first roadmap for client PCs today, in a bid to push into a larger number of systems and to challenge Intel for client computing dominance. While the company is still focused on mobile experiences, laptops dominate PC shipments today. The company is promising to deliver performance improvements of 15 percent per year or more through 2020. That rate of improvement would far outstrip anything we’ve seen from Intel or AMD in the past five years (if you don’t count the massive uplift AMD received when it transitioned off the equivalent of a silicon potato to its new Zen architecture).

ARM’s First Client PC Roadmap Makes Bold Claims, Doesn’t Back Them Up

What’s not clear, based solely on the data ARM has shown, is whether it can deliver what it promises. Let’s look at why. Each slide below can be expanded in a separate window by clicking on it.

It’s great to see ARM taking steps to be more transparent about its client roadmaps and projects. The company has clearly been angling to enter the client market more aggressively, as evidenced by the new crop of Windows on ARM devices. We appreciate that the company is being more communicative on these issues and we hope it continues. But as a question of whether ARM made a real case for itself with these slides? Not really. In fact, not even close.

First, ARM’s performance claims are based solely on its own estimate in a benchmark that’s susceptible to being gamed. Without knowing something about which settings were used to compile the benchmarks that make up SPECint 2006, we can’t judge the accuracy of ARM’s performance claims at all. Using Ubuntu 18.04 instead of Windows doesn’t make the results invalid, but it does make it more difficult to make assumptions about what Windows users will see when these devices hit store shelves. Given that x86 compatibility is provided via emulation (and matters to pretty much every Windows buyer), that’s a meaningful issue.

Second, ARM’s comparison graphs between its own performance trajectories and those of Intel have a massive omission: Coffee Lake. Intel’s decision to add more cores to its CPUs and hold TDPs the same had a significant and positive impact on overall performance. Benchmarking a dual-core Core i5-7300U as if the quad-core Core i5-8250U didn’t exist makes ARM look like it has something to hide — especially since the line graph of Intel SoCs versus ARM’s own products shows the Intel chips with much higher single-thread performance (check the fine print on Slide 5). Combine that advantage with the extra cores on a modern Intel mobile SoC, and all of a sudden the Cortex-A76 and its successors aren’t looking so good.

Third, using TDP as a point of comparison for power envelopes without additional information and context is misleading. Intel’s TDP figures are not a measure of power consumption but a value provided to thermal solution designers for heatsink design. It’s true that the Windows on ARM devices available in-market today draw less power in absolute terms than most, if not all, of Intel’s products — but you can’t simplify overall laptop power consumption with a reference to SoC TDP and then claim victory when you haven’t even clarified whether you’re measuring the same thing. Combine this with the issue of processor selection and the complete non-clarity around SPECint 2006 flags and none of the claims ARM makes can be meaningfully substantiated.

It’s very much worth asking whether ARM can build a CPU that’s intrinsically faster than Intel’s best and what that would mean for the semiconductor industry. We aren’t saying the company can’t. But if ARM wants to make the case that it’s going to deliver CPU architectures in 2019 and 2020 that are capable of beating the best the x86 manufacturers can offer in client computing workloads while running Windows 10, it’s going to have to offer much better evidence.