MediaTek caught gaming benchmark tests, says others do the same

Chipset company MediaTek caught gaming benchmarks with special functions in the firmware.

Can benchmark tests be trusted? (REUTERS)

Benchmark rigging is not a new concept. Smartphone companies have been doing it for years. Now, chipset company MediaTek has been caught gaming benchmark scores.

Anandtech has discovered MediaTek gaming benchmark scores of its Helio P95 chip that runs on Oppo Reno 3 Pro and Dimensity 1000L chip that runs on the China variant of the Oppo phone.

The publication in its report says it discovered the artificial optimisation when Reno 3 Pro outperformed Redno 3 in a popular PCMark benchmark test. It pointed out that P95 SoC used older generation Cortex CPU cores than the Dimensity 100L's.

The publication then installed a stealth version of the benchmark to test the performance. As it suspected, there was about a 30% drop in the benchmark score.

MediaTek, however, defended its approach towards benchmarking scores. In a blog post, MediaTek said it was a regular practice across the industry to optimise the performance for benchmarking tests.

"MediaTek follows accepted industry standards and is confident that benchmarking tests accurately represent the capabilities of our chipsets. We work closely with global device makers when it comes to testing and benchmarking devices powered by our chipsets, but ultimately brands have the flexibility to configure their own devices as they see fit. Many companies design devices to run on the highest possible performance levels when benchmarking tests are running in order to show the full capabilities of the chipset. This reveals what the upper end of performance capabilities are on any given chipset," it said in the post.

"Of course, in real world scenarios there are a multitude of factors that will determine how chipsets perform. MediaTek's chipsets are designed to optimize power and performance to provide the best user experience possible while maximizing battery life. If someone is running a compute-intensive program like a demanding game, the chipset will intelligently adapt to computing patterns to deliver sustained performance. This means that a user will see different levels of performance from different apps as the chipset dynamically manages the CPU, GPU and memory resources according to the power and performance that is required for a great user experience. Additionally, some brands have different types of modes turned on in different regions so device performance can vary based on regional market requirements," it added.

Benchmark rigging

As said earlier, smartphone companies have been gaming benchmark tests by optimising the performance especially for these tests. Over the years, many smartphone companies have been caught doing so. Many smartphone reviewers use these benchmark scores to base their opinion on the performance of a device. In some cases, topping benchmark rankings have also been used for marketing by some companies.