or Connect
AppleInsider › Forums › Mobile › iPhone › Galaxy S 4 on steroids: Samsung caught doping in benchmarks
New Posts  All Forums:Forum Nav:

Galaxy S 4 on steroids: Samsung caught doping in benchmarks

post #1 of 153
Thread Starter 
Tipped off by a report from hacker in Luxembourg that accused Samsung of increasing the clock speed of its Android-based Galaxy S 4 specifically, and only, when running certain benchmarks, AnandTech investigated and confirmed that it is indeed cheating to win in performance tests, by design.

Samsung


A report by Anand Lal Shimpi & Brian Klug examined versions of Samsung's Galaxy S 4 equipped with the company's new Exynos 5 Octa SoC.

Chip, set, match



Samsung announced the new Exynos 5 Octa component earlier this year as its answer to Apple's A6X; both chips are manufactured at Samsung's System LSI chip fab.

Apple uses its own custom "Swift" CPU core design, paired with GPU cores tech it licenses from Imagination Technologies, providing four PowerVR SGX 554MP4 cores to power graphics on iPad 4.

Samsung's Exynos 5 Octa was the first to license ARM's stock Cortex-A15 CPU core design, and pairs four of them with essentially the same Imagination GPU design as Apple, a shift from Samsung's earlier chips that formerly used ARM's Mali GPU design.


imagine


via Imagination.


Designed to win benchmarks



Their shared GPU design makes the two chips easier to compare, even though Samsung's are clocked to run faster (533MHz vs 300MHz).

However, in looking at Samsung's chip, AnandTech was able to verify what a poster had reported on graphics website Beyond3D: Samsung was specifically enabling full speed GPU performance only when running specific benchmarks, and then reverting to slower 480MHz speeds when being used to play games or run other apps.

June 27, 2013


The site first confirmed that, in normal use, the Galaxy S 4's Exynos 5 Octa was running at 480MHz, noting, "Samsung never publicly claimed max GPU frequencies for the Exynos 5 Octa (our information came from internal sources), so no harm no foul thus far."

However, when running tests including GLBenchmark 2.5.1, AnTuTu and Quadrant, the device "triggers a GPU clock not available elsewhere: 532MHz."

The CPU side behaved similarly when running benchmarks, but it used maximum performance modes available to other apps. "The 532MHz max GPU frequency on the other hand," the report stated, "is only available to these specific benchmarks."

Digging into files in the software that controls the chip's frequency, AnandTech discovered "what appeared to be hard coded profiles/exceptions for certain applications," including the text string "BenchmarkBooster."


Source: AnandTech


The file directly revealed custom Samsung support for changing how the Galaxy S 4 works when running specific benchmarks.

Further, the site discovered "there are strings for Fusion3 (the Snapdragon 600 + MDM9x15 combo) and Adonis (the codename for Exynos 5 Octa)," indicating that Samsung isn't just cheating on benchmarks for one of its processors; it's part of the company's culture.

Samsung: focus on user experience, not cheating at performance tests



The site noted, "you should be careful about comparing Exynos 5 Octa based Galaxy S 4s using any of the affected benchmarks to other devices and drawing conclusions based on that. This seems to be purely an optimization to produce repeatable (and high) results in CPU tests, and deliver the highest possible GPU performance benchmarks."

It further recommended, "what Samsung needs to do going forward is either open up these settings for all users/applications" or simply "remove the optimization altogether."

If Samsung keeps up software doping to run the Galaxy S 4 on steroids only when its being tested, the site warned that the industry will "end up in an arms race between all of the SoC and device makers where non-insignificant amounts of time and engineering effort is spent on gaming the benchmarks rather than improving user experience," something it noted has occured in the PC industry.
post #2 of 153
lol.
post #3 of 153

I'm shocked! /s

Life is too short to drink bad coffee.

Reply

Life is too short to drink bad coffee.

Reply
post #4 of 153
Wow. Thanks, Scamsung.
post #5 of 153
Samsung...Winning!
post #6 of 153

That's quite funny.


What a bunch of classless arseholes.

post #7 of 153
Why am I not surprised.
post #8 of 153
Why am I not surprised? scumbags I advise all my friends and family to avoid Samsung products I suggest we all do the same.

Originally by Rickers - 2014 : Cook & will bury Apple.  They can only ride Steve's ghost so long.


History reduce Apple Watch.... to a footnote in the annals of technology - Benjamin Frost Dec 2014



Reply

Originally by Rickers - 2014 : Cook & will bury Apple.  They can only ride Steve's ghost so long.


History reduce Apple Watch.... to a footnote in the annals of technology - Benjamin Frost Dec 2014



Reply
post #9 of 153
Unbelievable. Samsung has no shame. None what so ever

Windows survivor - after a long, epic and painful struggle. Very long AAPL

Reply

Windows survivor - after a long, epic and painful struggle. Very long AAPL

Reply
post #10 of 153

Class act!

post #11 of 153
Unscrupulous. Trying to get everyone to buy into their bull on specs and performance while selling a pinto engine in a mustang body. Lol.

Look!! Our screen is bigger than Apple's. Our processor is faster than Apple's. We are cool and Apple is for older folks. Lol. You can't use SD cards in Apple stuff, you can with Samsung (that's because we use over have your internal memory).

But people buy their stuff anyway and think they better than Apple and are somehow getting the best deal out there without paying a higher price.

You get what you pay for. Lol.

2010 15" MBP, iPhone 5 64GB, New iPad 64GB LTE, (2) ATV 2nd Gen

Reply

2010 15" MBP, iPhone 5 64GB, New iPad 64GB LTE, (2) ATV 2nd Gen

Reply
post #12 of 153

I'm just waiting for the troll who will say they see nothing wrong with this.

post #13 of 153
doesn't that amount to fraudulent advertising?
post #14 of 153
Quote:
Originally Posted by TheUnfetteredMind View Post

Class act!
Yeah and if Apple did this there would be a class action lawsuit.
post #15 of 153
Liars, cheats and scumbags! How shocking!
post #16 of 153
Quote:
Originally Posted by ifij775 View Post

Samsung...Winning!

S4s run on tiger blood, baby!

post #17 of 153
Everyone knew Exynos was trash but was somehow benchmarking well. Now we know the reason.
post #18 of 153

Stay classy, Samsung!

 

lol.gif

   Apple develops an improved programming language.  Google copied Java.  Everything you need to know, right there.

 

    AT&T believes their LTE coverage is adequate

Reply

   Apple develops an improved programming language.  Google copied Java.  Everything you need to know, right there.

 

    AT&T believes their LTE coverage is adequate

Reply
post #19 of 153
Well, at least Apple is trying to move there chips and screens away from samsung. Well a Galaxy S4 is still ---t to me either way.
Cheats
post #20 of 153
Do I smell a class action lawsuit in their future?
It is useless for sheep to pass laws outlawing carnivorism when the wolf is of a different mind.
Reply
It is useless for sheep to pass laws outlawing carnivorism when the wolf is of a different mind.
Reply
post #21 of 153

The Samsung 'Reputation Management' team is studying their scripts quickly, stand by for their BS explanations. 

iMac 2007, Macbook pro 2008, Mac Mini 2011
Reply
iMac 2007, Macbook pro 2008, Mac Mini 2011
Reply
post #22 of 153
Simple solution:

Aggregate Performance Measurement
  1. Start the benchmark on a device with a full battery charge
  2. Run benchmark until battery dies
  3. Normalize results to device with largest aggregate

In benchmarking as in lovemaking -- you don't just count the short strokes!
"Swift generally gets you to the right way much quicker." - auxio -

"The perfect [birth]day -- A little playtime, a good poop, and a long nap." - Tomato Greeting Cards -
Reply
"Swift generally gets you to the right way much quicker." - auxio -

"The perfect [birth]day -- A little playtime, a good poop, and a long nap." - Tomato Greeting Cards -
Reply
post #23 of 153

Now that's funny!  I see ROM threads on XDA all the time where users are comparing benchmark scores (as if it's relevant in some way).  Maybe this will help distance people from the idea that these test scores are valuable.  What's important is how the device operates everyday during normal use.

"Proof is irrelevant" - Solipsism
Reply
"Proof is irrelevant" - Solipsism
Reply
post #24 of 153
Quote:
Originally Posted by Feynman View Post


Yeah and if Apple did this there would be a class action lawsuit.

It would certainly be splashed all over the tech headlines if Apple did it. I guess we'll have to wait and see if Samsung gets the same treatment.

 

I don't think Apple would ever bother since they don't seem to truly care about the spec game. Frankly, I don't see why Samsung would even bother doing this. They're already crushing the rest of the Android competition and doing quite well vs Apple. I don't really see the payoff vs the risk of getting caught (like they now have been). Maybe they feel the risk (will people start to move to other manufacturers due to such activity?) is pretty low so, why not?

post #25 of 153
Quote:
Originally Posted by DroidFTW View Post

Now that's funny!  I see ROM threads on XDA all the time where users are comparing benchmark scores (as if it's relevant in some way).  Maybe this will help distance people from the idea that these test scores are valuable.  What's important is how the device operates everyday during normal use.

But benchmarks and spec sheets are all that matters! Not stupid things like UI responsiveness and how actual apps perform during real-world usage.

post #26 of 153


THE CLOCK RATE MAXES AT 480 INSTEAD OF 533.  NOT 300 LIKE APPLEINSIDER IS CLAIMING.

is it bad still, yes.  But not as bad as they are saying.

Quote:
Originally Posted by Phone-UI-Guy View Post

Why am I not surprised.

Probably because almost all companies do this in one way or another.

Quote:
Originally Posted by Paul94544 View Post

Why am I not surprised? scumbags I advise all my friends and family to avoid Samsung products I suggest we all do the same.

Why?  Almost every company does this for different benchmarks?  This would be like telling you family to avoid Intel because they do tons of optimizations when they run their benchmarks for their claims?

Quote:
Originally Posted by alandail View Post

doesn't that amount to fraudulent advertising?

No, it does not.  The chip runs at 533Mhz (well, 532 according to anandtech) during some benchmarks.  Otherwise it caps at 480Mhz.

There is nothing that talks about the specifications on the feature page, and the fine print would take care of it.

You technically can get the GPU to hit the rate they provided, just not in a useful manner.

Quote:
Originally Posted by Cyberzombie View Post

Do I smell a class action lawsuit in their future?

No.  At least, I don't see what it would be about.

You can still hit that clock rate, you just cannot do it in a useful manner.

-QAMF

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply
post #27 of 153

Oul Sammy, boy. Always honest.

Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #28 of 153
Quote:
Originally Posted by QAMF View Post


THE CLOCK RATE MAXES AT 480 INSTEAD OF 533.  NOT 300 LIKE APPLEINSIDER IS CLAIMING.

The 300mhz clock rate was for what the iPad 4's GPU cores run at. It was in reference to this section:

 

Quote:
Apple uses its own custom "Swift" CPU core design, paired with GPU cores tech it licenses from Imagination Technologies, providing four PowerVR SGX 554MP4 cores to power graphics on iPad 4.

Samsung's Exynos 5 Octa was the first to license ARM's stock Cortex-A15 CPU core design, and pairs four of them with essentially the same Imagination GPU design as Apple, a shift from Samsung's earlier chips that formerly used ARM's Mali GPU design.
 
...
 
Their shared GPU design makes the two chips easier to compare, even though Samsung's are clocked to run faster (533MHz vs 300MHz).

 

Basically they were saying that since both Apple and Samsung were using the same GPU cores it was simple to compare GPU performance between them since the difference was only in the clock rates (which was the reference of 533mhz vs 300mhz). Reading comprehension is a good skill you might want to pick up.

post #29 of 153
Not covered in this article is WHY Samsung would do this. The speed is there, why not use it for all applications? I'm sure the tech community will start guessing, and I have two possibilities:

1) When going full speed for extended lengths of time, the chip builds up too much heat and with either shut down or either damage other hardware of damage the chip itself.

2) When going full speed it drains the battery at an accelerated rate. The battery life would suffer so much the phone would be unsellable.
post #30 of 153
Quote:
Originally Posted by Don108 View Post

Not covered in this article is WHY Samsung would do this. The speed is there, why not use it for all applications? I'm sure the tech community will start guessing, and I have two possibilities:

1) When going full speed for extended lengths of time, the chip builds up too much heat and with either shut down or either damage other hardware of damage the chip itself.

2) When going full speed it drains the battery at an accelerated rate. The battery life would suffer so much the phone would be unsellable.

As you state: increased heat and faster battery drain.

post #31 of 153
Quote:
Originally Posted by MikeJones View Post

But benchmarks and spec sheets are all that matters! Not stupid things like UI responsiveness and how actual apps perform during real-world usage.

 

Spec sheets can absolutely be relevant.  It seems that those who want to ignore spec sheets are usually the ones who are lacking in said department.  Though obviously not the only metric to determine worth, they can be important.  They are related to how a device runs all applications.  Benchmarks on the other hand measure how well a device runs one application (the test itself).  I'm not sure I've ever found the value of a benchmark score outside of epeen contests.

"Proof is irrelevant" - Solipsism
Reply
"Proof is irrelevant" - Solipsism
Reply
post #32 of 153
Quote:
Originally Posted by Don108 View Post

Not covered in this article is WHY Samsung would do this. The speed is there, why not use it for all applications? I'm sure the tech community will start guessing, and I have two possibilities:

1) When going full speed for extended lengths of time, the chip builds up too much heat and with either shut down or either damage other hardware of damage the chip itself.

2) When going full speed it drains the battery at an accelerated rate. The battery life would suffer so much the phone would be unsellable.

 

I'd guess number 2 is the most likely.

post #33 of 153
I'm waiting for Gatorguy and KDarling to weigh in on this issue!
1biggrin.gif

We've always been at war with Eastasia...

Reply

We've always been at war with Eastasia...

Reply
post #34 of 153
Quote:
Originally Posted by QAMF View Post


THE CLOCK RATE MAXES AT 480 INSTEAD OF 533.  NOT 300 LIKE APPLEINSIDER IS CLAIMING.

is it bad still, yes.  But not as bad as they are saying.

Probably because almost all companies do this in one way or another.

Why?  Almost every company does this for different benchmarks?  This would be like telling you family to avoid Intel because they do tons of optimizations when they run their benchmarks for their claims?

No, it does not.  The chip runs at 533Mhz (well, 532 according to anandtech) during some benchmarks.  Otherwise it caps at 480Mhz.

There is nothing that talks about the specifications on the feature page, and the fine print would take care of it.

You technically can get the GPU to hit the rate they provided, just not in a useful manner.

No.  At least, I don't see what it would be about.

You can still hit that clock rate, you just cannot do it in a useful manner.

-QAMF

 

Predatory ethics!  I guess honesty is just for the weak minded.

post #35 of 153
Quote:
Originally Posted by DroidFTW View Post

 

Spec sheets can absolutely be relevant.  It seems that those who want to ignore spec sheets are usually the ones who are lacking in said department.  Though obviously not the only metric to determine worth, they can be important.  I'm not sure I've ever found the value of a benchmark score outside of epeen contests.

They can be relevant for certain things like display quality, audio quality, etc. In most cases, though, spec sheets are thrown out as a way to distract you from the fact that the performance of the software in the system is so abysmal that they had to have such high specs just to make it run decently. In fact, a spec sheet is more impressive if you show that you are doing just as much, if not possibly more, than your competition with lower speced devices than theirs. If my phone with half the RAM and half the CPU cores has better UI responsiveness, runs all the same apps and games, and has equally if not better performance in real-world usage how exactly is it "lacking" anything? Going around bragging to everyone about the specs of your phone is just as much epeen waving as benchmark scores.

post #36 of 153
Quote:
Originally Posted by Don108 View Post

Not covered in this article is WHY Samsung would do this. The speed is there, why not use it for all applications? I'm sure the tech community will start guessing, and I have two possibilities:

1) When going full speed for extended lengths of time, the chip builds up too much heat and with either shut down or either damage other hardware of damage the chip itself.

2) When going full speed it drains the battery at an accelerated rate. The battery life would suffer so much the phone would be unsellable.

 

If you're truly interested in the answer, you should check out some introductory writeups about the pros and cons of over-clocking.

"Proof is irrelevant" - Solipsism
Reply
"Proof is irrelevant" - Solipsism
Reply
post #37 of 153
That's why the Samsung S4 could run so damn HOT and burnt down people house.

See the HK local news from last Saturday, a guy playing game and charging his Samsung S4 at the same time, the Samsung S4 exploded and burnt down his house.
http://translate.google.com/translate?sl=auto&tl=en&js=n&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fthe-sun.on.cc%2Fcnt%2Fnews%2F20130727%2F00407_006.html&act=url
Edited by marsk - 7/30/13 at 11:06am
post #38 of 153
This doesn't shock me one iota.

What also doesn't shock me is this news won't make a blip in mainstream news. If Apple did something like this, it would be slathered all over major blog and news sites like CNN with headlines to match. I've yet to see SAmsung being honest about a single aspect of their products or business. But hey, apparently they're the "next Apple" or something, and we're supposed to cheer them on regardless.
post #39 of 153
Quote:
Originally Posted by Slurpy View Post

This doesn't shock me one iota.

What also doesn't shock me is this news won't make a blip in mainstream news. If Apple did something like this, it would be slathered all over major blog and news sites like CNN with headlines to match. I've yet to see SAmsung being honest about a single aspect of their products or business. But hey, apparently they're the "next Apple" or something, and we're supposed to cheer them on regardless.

But the Samsung apologists tell us that everyone does it. So that makes it's okay!

post #40 of 153
Quote:
Originally Posted by QAMF View Post

No, it does not.  The chip runs at 533Mhz (well, 532 according to anandtech) during some benchmarks.  Otherwise it caps at 480Mhz.

There is nothing that talks about the specifications on the feature page, and the fine print would take care of it.

 

I completely disagree.  Benchmarks numbers are used to advertise your product as faster than your competitor for running apps.  Over clocking your processor only to run the benchmark, and not to run any other apps is flat out intentional fraudulent advertising.

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Galaxy S 4 on steroids: Samsung caught doping in benchmarks