UL Delists Huawei Benchmark Scores After Cheating Discovered

UL Delists Huawei Benchmark Scores After Cheating Discovered

UL, which owns 3DMark, has issued its own statement on Huawei’s benchmarking practices after the company was discovered to be artificially inflating its benchmark performance by pushing its SoCs out of thermal bounds when benchmarks were added to a whitelist that allowed the SoC to run at clocks that actually match those listed on a phone’s packaging. In all other cases, devices run far below their listed clocks to stay within thermal limits.

Huawei may have come up with this idea as a means to avoid being accused of overclocking its hardware — you can’t argue something is overclocked if it’s only hitting its specified frequencies — but the larger problem is that the company’s hardware tweaks completely blow the power budget and only apply to whitelisted benchmarks.

Data by Anandtech
Data by Anandtech

Huawei and UL have had several conversations on this topic in which Huawei attempted to explain that this wasn’t cheating, but reflected new AI optimizations being used to ensure maximum performance. At the same time, however, these AI optimizations are only being used when the handset is benchmarked in specific tests as opposed to in games — and they’re pushing the devices so far out of bounds, they’re overheating in some cases. Since the entire point of a benchmark is to capture useful information about the performance of a device, a benchmark mode that only runs when the device is being tested is the exact opposite of helpful.

UL writes:

UL understands the intent of Huawei’s approach, but is opposed to forcing the use of a “Performance Mode” by default when a benchmarking application is detected by the device. UL rules require a device to run the benchmark as if it were any other application.

Huawei respects consumers’ right to choose what to do with their devices. Therefore, Huawei will provide users with open access to “Performance Mode” in EMUI 9.0, so that the user can choose when to use the maximum power of their device…

To prevent confusion around current benchmarking results, after discussion, UL and Huawei have temporarily delisted the benchmark scores of a range of Huawei devices, and will reinstate them after Huawei grants all users of Huawei handsets access to the Performance Mode.

This kind of problem is far from new in mobile benchmarking, and while there’s evidence that Huawei is indeed trying to develop a new approach to improving its GPU performance through the use of AI, what the company did here appears far more like a standard whitelist trick to boost performance in the apps people use to test phones by blowing the SoC’s power curve. The device can’t practically operate for any length of time while drawing 8W of power, and that’s what matters — far more than any question of AI optimization or feature usefulness.

Continue reading

Huawei’s Phone Deal With AT&T Reportedly Killed On Account of Politics
Huawei’s Phone Deal With AT&T Reportedly Killed On Account of Politics

The upcoming (and unannounced) deal with AT&T to sell the new Mate 10 series was supposed to be the start of Huawei's push into North America, but the deal has reportedly fallen apart at the last minute after AT&T got cold feet, and some sources point to a political cause.

Lawmakers Urge AT&T to Cut Ties with Huawei, Citing National Security Concerns
Lawmakers Urge AT&T to Cut Ties with Huawei, Citing National Security Concerns

It's been several years since the last dust-up, but US lawmakers and regulators are still sounding the alarm about any cooperation with Huawei.

ET Deals Roundup: $150 Gift Card on Huawei Mate 10 Pre-Orders, XPS 6-Core GTX 1080 Gaming Tower, and more
ET Deals Roundup: $150 Gift Card on Huawei Mate 10 Pre-Orders, XPS 6-Core GTX 1080 Gaming Tower, and more

Want a high-end gaming rig that's ready for true 4K gaming? Right now, Dell is offering up a new XPS configuration with a Coffee Lake six-core CPU, a GTX 1080 graphics card, and a 512GB SSD for under $1.5K. We've also found excellent deals on the likes of a Raspberry Pi 3 starter kit, the 7-inch Amazon Echo Show, and the upcoming Huawei Mate 10 Pro smartphone.

Huawei Had Fans Leave Over 100 Fake Reviews for the Mate 10 Pro
Huawei Had Fans Leave Over 100 Fake Reviews for the Mate 10 Pro

In a misguided attempt to hype up the phone before launch, Huawei appears to have solicited fake reviews on the Best Buy pre-order page in a foolishly transparent way.