Benchmarks are important metric by which the performance of a given device can be measured. They provide a reliable means of determining whether one laptop, computer, phone or other device is faster than another — at least that’s the idea. OnePlus and Meizu are manipulating results of benchmark tests, says a report by XDA Developers. Specifically, the OnePlus 3T and the Meizu Pro 6 have been found guilty. According to XDA’s report, these smartphones detect when benchmark tests are being run and perform differently in order to produce better benchmark results.
XDA noted than when opening certain benchmarking apps, the OnePlus 3T’s cores would stay above 0.98GHz for the little cores and 1.29GHz for the big cores. Usually, both cores drop to 0.31GHz when there is no load. It seems the OnePlus 3T was targeting specific benchmarks by name and was entering a different mode, unknown to users, in order to increase scores. XDA partnered with Primate Labs, the creators of Geekbench, who provided them with a different version of the benchmark. This version was designed to avoid any benchmark detection and run the app as normal.
Our editor-in-chief, Mario Serrafero, was using Qualcomm Trepn and the Snapdragon Performance Visualizer to monitor how Qualcomm “boosts” the CPU clock speed when opening apps, and noticed that certain apps on the OnePlus 3T were not falling back down to their normal idling speeds after opening.
The website noted that the difference in individual runs on the OnePlus 3T were not very high. However the Meizu Pro 6 was found to be using similar techniques to inflate benchmark results and the difference in scores were more pronounced in the Meizu Pro 6. XDA noted that Meizu seems to be tuning its device to detect benchmarking apps and kick into a higher gear. Meizu often doesn’t allow the power intensive cores on its SoCs come alive, whereas, in this case they’re being used to the fullest. The graphs above and below were presented by XDA to show their findings.
When XDA reached out to them about the issue and asked them to address it, they said that this feature is in OxygenOS to help games and other performance-heavy apps perform better on their phones and that targeting benchmarking applications will be removed.
‘In order to give users a better user experience in resource intensive apps and games, especially graphically intensive ones, we implemented certain mechanisms in the community and Nougat builds to trigger the processor to run more aggressively. The trigger process for benchmarking apps will not be present in upcoming OxygenOS builds on the OnePlus 3 and OnePlus 3T.’
To their credit, OnePlus acknowledged their mistake and promised to fix it. XDA says this behavior was not present in the firmware when the OnePlus 3 launched and that helps making the company’s statement a bit more believable.
Nevertheless, there is no justification for deceptive behavior that ultimately hurts consumers. There’s a twist too and the publication reports that it tested other devices from multiple manufacturers and found some that failed the hidden benchmark test that exposed the OnePlus 3 and Meizu Pro 6. They haven’t revealed the names of these devices, pending more analysis but they did reveal the manufacturers that were not caught cheating: HTC, Xiaomi, Huawei, Honor, Google, and Sony. This still leaves out Samsung which was involved in a similar case along with most other manufacturers back in 2013.
0 comments:
Post a Comment