UL Delists Huawei Devices Due To Inaccurate Benchmarks
by Andrei Frumusanu on September 6, 2018 12:55 PM EST- Posted in
- Mobile
- Futuremark
- Smartphones
- 3DMark
- UL
A few days ago we published our article addressing Huawei and Honor’s inaccurate benchmarking behaviour. In a nutshell, we had found out that this year’s devices had introduced a new thermal limiting behaviour that quickly throttled power consumption to ~4-4.5W in 3D workloads. While this in itself wasn’t an issue, the problem is that the firmware did not apply this new behaviour to a specific list of whitelisted 3D benchmarks.
Sidenote: Earlier this year, Futuremark (The 3DMark creators) had been acquired by UL, joining the global safety consulting and certification company as a benchmarking division.
We work closely with all benchmark vendors, and UL isn’t an exception. We had first given UL note of the behaviour two weeks ago and had been sharing our early results with the development team. Today UL published their independent confirmation of our results on their own devices, and have subsequently decided to remove the tested devices from their results database.
UL stresses that the kind of detection and optimization performed by Huawei infringes the company’s rules for manufacturers. Unfortunately this isn’t the first instance of a vendor being delisted, as most famously a slew of phone manufacturer had been delisted in 2013 after a more in-depth investigation of ours resulted in quite embarrassing results for a lot of vendors.
The UL team further explains that they’ve also been in touch with Huawei, and that the Chinese vendor is planning to address the behaviour by introducing a new “performance mode” that disables the new thermal throttling behaviour. In essence this mode would revert back to the behaviour we’ve seen in the past such as the Mate 9 – where the SoC is allowed higher peak performance figures at a cost of high power.
While this is still problematic for representative benchmarking for devices with Kirin 960’s and 970’s (due to the very large peak power at the highest performance states of those SoCs), I expect this to be a non-issue for the new Kirin 980, as it’s projected to have significantly better sustainable peak power figures. We're still looking forward to the new SoC in the upcoming Mate 20, and believe it to be an outstanding performer.
Source: UL News Release
Facebook
Twitter
RSS
33 Comments
View All Comments
zmeul - Thursday, September 6, 2018 - link
geez fucking H christ with capitalizing each 1st letter of the title - who / why the hell do you do this?! I had to read the god darn thing twice because I could not understand itthis is how it should be:
UL delists Huawei devices due to inaccurate results
A5 - Thursday, September 6, 2018 - link
That's standard English formatting for titles/headlines?Maltz - Thursday, September 6, 2018 - link
Except for the "to", yeah.DanNeely - Thursday, September 6, 2018 - link
I assume you have problems reading articles almost any and everywhere else. Capitalizing the first letter of every word in a title (for a book, article, etc) has been a convention in English long enough that it's called title case. It's nothing new, but has been around for decades/(centuries?) as a way to emphasize words because traditional printing had far fewer ways to do so without additional expense (eg additional sets of physical letters in different sizes/fonts/weights).PeachNCream - Thursday, September 6, 2018 - link
The lack of any attempt at proper punctuation or capitalization in zmeul's post and the sprinkling of irrational profanity should speak for itself.Samus - Thursday, September 6, 2018 - link
^^^PeachNCream - Thursday, September 6, 2018 - link
AT's title is acceptable. Your recommendation is not.See: http://grammar.yourdictionary.com/capitalization/r...
seamonkey79 - Thursday, September 6, 2018 - link
If You Knew How English Titles Worked You Would Not Have a Problem - or That Is How It Should Be Everywhere.damianrobertjones - Thursday, September 6, 2018 - link
You should always start a sentence with a capital letter. Always. Please remember this, while on the web, or stating that someone has made a grammatical error.damianrobertjones - Thursday, September 6, 2018 - link
P.s. I'm seeing less and less posts without capitals. It's really annoying!