UL Delists Huawei Benchmark Scores After Cheating Discovered
UL, which owns 3DMark, has issued its own statement on Huawei’s benchmarking practices after the company was discovered to be artificially inflating its benchmark performance by pushing its SoCs out of thermal bounds when benchmarks were added to a whitelist that allowed the SoC to run at clocks that actually match those listed on a phone’s packaging. In all other cases, devices run far below their listed clocks to stay within thermal limits.
Huawei may have come up with this idea as a means to avoid being accused of overclocking its hardware — you can’t argue something is overclocked if it’s only hitting its specified frequencies — but the larger problem is that the company’s hardware tweaks completely blow the power budget and only apply to whitelisted benchmarks.
Huawei and UL have had several conversations on this topic in which Huawei attempted to explain that this wasn’t cheating, but reflected new AI optimizations being used to ensure maximum performance. At the same time, however, these AI optimizations are only being used when the handset is benchmarked in specific tests as opposed to in games — and they’re pushing the devices so far out of bounds, they’re overheating in some cases. Since the entire point of a benchmark is to capture useful information about the performance of a device, a benchmark mode that only runs when the device is being tested is the exact opposite of helpful.
UL writes:
UL understands the intent of Huawei’s approach, but is opposed to forcing the use of a “Performance Mode” by default when a benchmarking application is detected by the device. UL rules require a device to run the benchmark as if it were any other application.
Huawei respects consumers’ right to choose what to do with their devices. Therefore, Huawei will provide users with open access to “Performance Mode” in EMUI 9.0, so that the user can choose when to use the maximum power of their device…
To prevent confusion around current benchmarking results, after discussion, UL and Huawei have temporarily delisted the benchmark scores of a range of Huawei devices, and will reinstate them after Huawei grants all users of Huawei handsets access to the Performance Mode.
This kind of problem is far from new in mobile benchmarking, and while there’s evidence that Huawei is indeed trying to develop a new approach to improving its GPU performance through the use of AI, what the company did here appears far more like a standard whitelist trick to boost performance in the apps people use to test phones by blowing the SoC’s power curve. The device can’t practically operate for any length of time while drawing 8W of power, and that’s what matters — far more than any question of AI optimization or feature usefulness.
Continue reading
Huawei Sells Honor Brand Amid Tightening Trade Restrictions
(Credit: Kevin Frayer/Getty Images)Huawei has been battered by US trade restrictions in the last few years, and it’s taking a toll on the company’s long-term stability. Experts don’t expect a radical change when the new US administration comes to power next year, so Huawei is beginning to take drastic action. It has sold its Honor…
It Turns Out Huawei’s HarmonyOS Is Still Just Android
Following the Commerce Department's actions against the Chinese megafirm, Huawei has been unable to use Google services on its new phones. The company's solution was to develop HarmonyOS, but now that we've gotten our first real look at it, one thing is clear: this is just Android with a skin.
Huawei 2.0: Invading Ukraine Could Cost Russia the Modern Semiconductor Market
The United States is threatening to use export controls to sanction Russia if it invades Ukraine. The Biden Administration would presumably use some of the same tools the Trump Administration previously deployed against Huawei.The Ultimate BanhammerThe United States maintains a list of individuals, corporations, governments, and non-governmental organizations (NGOs) that are subject to export restrictions.…
Canada Bans ZTE and Huawei Network Infrastructure Citing National Security Concerns
Telecom companies in Canada will no longer be able to purchase infrastructure from the companies, and they will have to remove what they have in the coming years.