Galaxy S 4 on steroids: Samsung caught doping in benchmarks

Posted:
in iPhone edited January 2014
Tipped off by a report from hacker in Luxembourg that accused Samsung of increasing the clock speed of its Android-based Galaxy S 4 specifically, and only, when running certain benchmarks, AnandTech investigated and confirmed that it is indeed cheating to win in performance tests, by design.

Samsung


A report by Anand Lal Shimpi & Brian Klug examined versions of Samsung's Galaxy S 4 equipped with the company's new Exynos 5 Octa SoC.

Chip, set, match

Samsung announced the new Exynos 5 Octa component earlier this year as its answer to Apple's A6X; both chips are manufactured at Samsung's System LSI chip fab.

Apple uses its own custom "Swift" CPU core design, paired with GPU cores tech it licenses from Imagination Technologies, providing four PowerVR SGX 554MP4 cores to power graphics on iPad 4.

Samsung's Exynos 5 Octa was the first to license ARM's stock Cortex-A15 CPU core design, and pairs four of them with essentially the same Imagination GPU design as Apple, a shift from Samsung's earlier chips that formerly used ARM's Mali GPU design.


imagine


via Imagination.

Designed to win benchmarks

Their shared GPU design makes the two chips easier to compare, even though Samsung's are clocked to run faster (533MHz vs 300MHz).

However, in looking at Samsung's chip, AnandTech was able to verify what a poster had reported on graphics website Beyond3D: Samsung was specifically enabling full speed GPU performance only when running specific benchmarks, and then reverting to slower 480MHz speeds when being used to play games or run other apps.

@andreif7 thanks for sending that my way, I wasn?t aware. Really sad to see samsung doing that.

? Brian Klug (@nerdtalker)


The site first confirmed that, in normal use, the Galaxy S 4's Exynos 5 Octa was running at 480MHz, noting, "Samsung never publicly claimed max GPU frequencies for the Exynos 5 Octa (our information came from internal sources), so no harm no foul thus far."

However, when running tests including GLBenchmark 2.5.1, AnTuTu and Quadrant, the device "triggers a GPU clock not available elsewhere: 532MHz."

The CPU side behaved similarly when running benchmarks, but it used maximum performance modes available to other apps. "The 532MHz max GPU frequency on the other hand," the report stated, "is only available to these specific benchmarks."

Digging into files in the software that controls the chip's frequency, AnandTech discovered "what appeared to be hard coded profiles/exceptions for certain applications," including the text string "BenchmarkBooster."


Source: AnandTech


The file directly revealed custom Samsung support for changing how the Galaxy S 4 works when running specific benchmarks.

Further, the site discovered "there are strings for Fusion3 (the Snapdragon 600 + MDM9x15 combo) and Adonis (the codename for Exynos 5 Octa)," indicating that Samsung isn't just cheating on benchmarks for one of its processors; it's part of the company's culture.

Samsung: focus on user experience, not cheating at performance tests

The site noted, "you should be careful about comparing Exynos 5 Octa based Galaxy S 4s using any of the affected benchmarks to other devices and drawing conclusions based on that. This seems to be purely an optimization to produce repeatable (and high) results in CPU tests, and deliver the highest possible GPU performance benchmarks."

It further recommended, "what Samsung needs to do going forward is either open up these settings for all users/applications" or simply "remove the optimization altogether."

If Samsung keeps up software doping to run the Galaxy S 4 on steroids only when its being tested, the site warned that the industry will "end up in an arms race between all of the SoC and device makers where non-insignificant amounts of time and engineering effort is spent on gaming the benchmarks rather than improving user experience," something it noted has occured in the PC industry.
«1345678

Comments

  • Reply 1 of 153
    tallest skiltallest skil Posts: 43,399member
    lol.
  • Reply 2 of 153
    mstonemstone Posts: 11,510member


    I'm shocked! /s

  • Reply 3 of 153
    grbladegrblade Posts: 92member
    Wow. Thanks, Scamsung.
  • Reply 4 of 153
    ifij775ifij775 Posts: 470member
    Samsung...Winning!
  • Reply 5 of 153
    paulmjohnsonpaulmjohnson Posts: 1,380member


    That's quite funny.




    What a bunch of classless arseholes.

  • Reply 6 of 153
    phone-ui-guyphone-ui-guy Posts: 1,019member
    Why am I not surprised.
  • Reply 7 of 153
    paul94544paul94544 Posts: 1,027member
    Why am I not surprised? scumbags I advise all my friends and family to avoid Samsung products I suggest we all do the same.
  • Reply 8 of 153
    red oakred oak Posts: 900member
    Unbelievable. Samsung has no shame. None what so ever
  • Reply 9 of 153


    Class act!

  • Reply 10 of 153
    pujones1pujones1 Posts: 222member
    Unscrupulous. Trying to get everyone to buy into their bull on specs and performance while selling a pinto engine in a mustang body. Lol.

    Look!! Our screen is bigger than Apple's. Our processor is faster than Apple's. We are cool and Apple is for older folks. Lol. You can't use SD cards in Apple stuff, you can with Samsung (that's because we use over have your internal memory).

    But people buy their stuff anyway and think they better than Apple and are somehow getting the best deal out there without paying a higher price.

    You get what you pay for. Lol.
  • Reply 11 of 153
    genovellegenovelle Posts: 1,296member


    I'm just waiting for the troll who will say they see nothing wrong with this.

  • Reply 12 of 153
    alandailalandail Posts: 696member
    doesn't that amount to fraudulent advertising?
  • Reply 13 of 153
    feynmanfeynman Posts: 1,087member
    Class act!
    Yeah and if Apple did this there would be a class action lawsuit.
  • Reply 14 of 153
    apple ][apple ][ Posts: 9,233member
    Liars, cheats and scumbags! How shocking!
  • Reply 15 of 153
    mikejonesmikejones Posts: 323member

    Quote:

    Originally Posted by ifij775 View Post



    Samsung...Winning!


    S4s run on tiger blood, baby!

  • Reply 16 of 153
    Everyone knew Exynos was trash but was somehow benchmarking well. Now we know the reason.
  • Reply 17 of 153
    john.bjohn.b Posts: 2,740member


    Stay classy, Samsung!


     


    image

  • Reply 18 of 153
    georgeip5georgeip5 Posts: 225member
    Well, at least Apple is trying to move there chips and screens away from samsung. Well a Galaxy S4 is still ---t to me either way.
    Cheats
  • Reply 19 of 153
    Do I smell a class action lawsuit in their future?
  • Reply 20 of 153


    The Samsung 'Reputation Management' team is studying their scripts quickly, stand by for their BS explanations. 

Sign In or Register to comment.