Galaxy S 4 on steroids: Samsung caught doping in benchmarks
Tipped off by a report from hacker in Luxembourg that accused Samsung of increasing the clock speed of its Android-based Galaxy S 4 specifically, and only, when running certain benchmarks, AnandTech investigated and confirmed that it is indeed cheating to win in performance tests, by design.
A report by Anand Lal Shimpi & Brian Klug examined versions of Samsung's Galaxy S 4 equipped with the company's new Exynos 5 Octa SoC.
Apple uses its own custom "Swift" CPU core design, paired with GPU cores tech it licenses from Imagination Technologies, providing four PowerVR SGX 554MP4 cores to power graphics on iPad 4.
Samsung's Exynos 5 Octa was the first to license ARM's stock Cortex-A15 CPU core design, and pairs four of them with essentially the same Imagination GPU design as Apple, a shift from Samsung's earlier chips that formerly used ARM's Mali GPU design.
However, in looking at Samsung's chip, AnandTech was able to verify what a poster had reported on graphics website Beyond3D: Samsung was specifically enabling full speed GPU performance only when running specific benchmarks, and then reverting to slower 480MHz speeds when being used to play games or run other apps.
The site first confirmed that, in normal use, the Galaxy S 4's Exynos 5 Octa was running at 480MHz, noting, "Samsung never publicly claimed max GPU frequencies for the Exynos 5 Octa (our information came from internal sources), so no harm no foul thus far."
However, when running tests including GLBenchmark 2.5.1, AnTuTu and Quadrant, the device "triggers a GPU clock not available elsewhere: 532MHz."
The CPU side behaved similarly when running benchmarks, but it used maximum performance modes available to other apps. "The 532MHz max GPU frequency on the other hand," the report stated, "is only available to these specific benchmarks."
Digging into files in the software that controls the chip's frequency, AnandTech discovered "what appeared to be hard coded profiles/exceptions for certain applications," including the text string "BenchmarkBooster."
Source: AnandTech
The file directly revealed custom Samsung support for changing how the Galaxy S 4 works when running specific benchmarks.
Further, the site discovered "there are strings for Fusion3 (the Snapdragon 600 + MDM9x15 combo) and Adonis (the codename for Exynos 5 Octa)," indicating that Samsung isn't just cheating on benchmarks for one of its processors; it's part of the company's culture.
It further recommended, "what Samsung needs to do going forward is either open up these settings for all users/applications" or simply "remove the optimization altogether."
If Samsung keeps up software doping to run the Galaxy S 4 on steroids only when its being tested, the site warned that the industry will "end up in an arms race between all of the SoC and device makers where non-insignificant amounts of time and engineering effort is spent on gaming the benchmarks rather than improving user experience," something it noted has occured in the PC industry.
A report by Anand Lal Shimpi & Brian Klug examined versions of Samsung's Galaxy S 4 equipped with the company's new Exynos 5 Octa SoC.
Chip, set, match
Samsung announced the new Exynos 5 Octa component earlier this year as its answer to Apple's A6X; both chips are manufactured at Samsung's System LSI chip fab.Apple uses its own custom "Swift" CPU core design, paired with GPU cores tech it licenses from Imagination Technologies, providing four PowerVR SGX 554MP4 cores to power graphics on iPad 4.
Samsung's Exynos 5 Octa was the first to license ARM's stock Cortex-A15 CPU core design, and pairs four of them with essentially the same Imagination GPU design as Apple, a shift from Samsung's earlier chips that formerly used ARM's Mali GPU design.
Designed to win benchmarks
Their shared GPU design makes the two chips easier to compare, even though Samsung's are clocked to run faster (533MHz vs 300MHz).However, in looking at Samsung's chip, AnandTech was able to verify what a poster had reported on graphics website Beyond3D: Samsung was specifically enabling full speed GPU performance only when running specific benchmarks, and then reverting to slower 480MHz speeds when being used to play games or run other apps.
@andreif7 thanks for sending that my way, I wasn?t aware. Really sad to see samsung doing that.
? Brian Klug (@nerdtalker)
The site first confirmed that, in normal use, the Galaxy S 4's Exynos 5 Octa was running at 480MHz, noting, "Samsung never publicly claimed max GPU frequencies for the Exynos 5 Octa (our information came from internal sources), so no harm no foul thus far."
However, when running tests including GLBenchmark 2.5.1, AnTuTu and Quadrant, the device "triggers a GPU clock not available elsewhere: 532MHz."
The CPU side behaved similarly when running benchmarks, but it used maximum performance modes available to other apps. "The 532MHz max GPU frequency on the other hand," the report stated, "is only available to these specific benchmarks."
Digging into files in the software that controls the chip's frequency, AnandTech discovered "what appeared to be hard coded profiles/exceptions for certain applications," including the text string "BenchmarkBooster."
Source: AnandTech
The file directly revealed custom Samsung support for changing how the Galaxy S 4 works when running specific benchmarks.
Further, the site discovered "there are strings for Fusion3 (the Snapdragon 600 + MDM9x15 combo) and Adonis (the codename for Exynos 5 Octa)," indicating that Samsung isn't just cheating on benchmarks for one of its processors; it's part of the company's culture.
Samsung: focus on user experience, not cheating at performance tests
The site noted, "you should be careful about comparing Exynos 5 Octa based Galaxy S 4s using any of the affected benchmarks to other devices and drawing conclusions based on that. This seems to be purely an optimization to produce repeatable (and high) results in CPU tests, and deliver the highest possible GPU performance benchmarks."It further recommended, "what Samsung needs to do going forward is either open up these settings for all users/applications" or simply "remove the optimization altogether."
If Samsung keeps up software doping to run the Galaxy S 4 on steroids only when its being tested, the site warned that the industry will "end up in an arms race between all of the SoC and device makers where non-insignificant amounts of time and engineering effort is spent on gaming the benchmarks rather than improving user experience," something it noted has occured in the PC industry.
Comments
I'm shocked! /s
That's quite funny.
What a bunch of classless arseholes.
Class act!
Look!! Our screen is bigger than Apple's. Our processor is faster than Apple's. We are cool and Apple is for older folks. Lol. You can't use SD cards in Apple stuff, you can with Samsung (that's because we use over have your internal memory).
But people buy their stuff anyway and think they better than Apple and are somehow getting the best deal out there without paying a higher price.
You get what you pay for. Lol.
I'm just waiting for the troll who will say they see nothing wrong with this.
Quote:
Originally Posted by ifij775
Samsung...Winning!
S4s run on tiger blood, baby!
Stay classy, Samsung!
Cheats
The Samsung 'Reputation Management' team is studying their scripts quickly, stand by for their BS explanations.