or Connect
AppleInsider › Forums › Mobile › iPhone › Galaxy S 4 on steroids: Samsung caught doping in benchmarks
New Posts  All Forums:Forum Nav:

Galaxy S 4 on steroids: Samsung caught doping in benchmarks - Page 4

post #121 of 153
Quote:
Originally Posted by macxpress View Post

Yeah, this sucks and everything, but most people won't care when they come in to buy one of these. Its not like they bought it because its supposed graphics powerhouse in the first place. 

This is only in the International Exynos 5410 version of the Galaxy S4, NOT the Snapdragon 600 model.

 

It would also be worth pointing out that even without the overclock, the Exynos 5410 still offers double the performance of the A6 inside of the iPhone 5, and both are behind the Snapdragon 600.

 

GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p):

 

Apple A6 - 373 Frames

 

Samsung Exynos 5410 - 794 Frames

 

Qualcomm Snapdragon 600 - 971 Frames

post #122 of 153
Quote:
Originally Posted by MikeJones View Post

1) So what? How does that absolve Samsung of any guilt?

2) They could but do you have any proof that they would?

 

And, yes, you are defending Samsung. Playing the "this isn't so bad because [insert other person/company/thing] did [insert thing they did] and it's worse!!" No one here has defended what Intel did so what is the point of bringing it up other than to deflect from Samsung's guilt?

1.  It does not.  However, if you want to complain about people and benchmarks, complain about Intel.  I am saying that people are making a huge deal out of something that in the grand scheme of benchmark "doping" is small.
2. would they? That depends on how bad public opinion gets.  Even Apple, who is one of the best companies in the world when it comes to PR had to admit fault when public opinion got bad enough for a problem that was super-rare.  This would require fixing the "problem" which Samsung would most likely opt to do.  From a PR perspective.

I am not defending what Samsung did.  I am saying that people should look at he larger scope.  I am saying on a scale of "how bad was the bad thing" Samsung ranks very lower.  That is not to say what Samsung did should be defended, that is to say that many companies that do worse.

Where was Apple Insider's story when Intel and Antutu happened?  The story that talked about how x86 was going to surpass ARM for power efficiency, which does matter to what SoC Apple could possibly use. (Intel was willing to license Atom in the past, had no takers.  Apple could do custom Atom if it was faster at the given power envelope).   Something that was far worse and not fixable (at the same power envelope, I suppose you could increase the clocks of the Atom by 20% or more, but that would drastically raise battery life) https://www.google.com/search?q=antutu+Intel&oq=antutu+Intel&aqs=chrome.0.69i57j0l3j69i62l2.2090j0&sourceid=chrome&ie=UTF-8

If you dislike this kind of stuff than go after everyone.  On the scale of "how badly _____ company cheat(s/ed) benchmarks" Samsung is not very high up.  Yet people are making a huge deal of out it.  What Samsung did is wrong.  It is "less wrong" than what Intel continues to do, and Nvidia to a lesser extent than Intel.  It is just what most companies that make their own hardware do.

-QAMF

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply
post #123 of 153
Quote:
Originally Posted by Dick Applebaum View Post

Funny... This article reminded me of a song:

I figured I could have some fun changing a few words in the lyrics,,,

]

How's this?

NEIL DIAMOND - SONG SUNG BLUE LYRICS

Samsung's screwed
Now that they've been busted
Samsung's screwed
Cause they can't be trusted

Samsung's Lies Are Subject To The Blues Now And Then
But When You Take The Lies And Publicize
You Might Get Caught Again
You Will get Caught Again

Samsung's Screwed
Weeping like a willow
Samsung's Screwed
Hear Them Pay Their Shill-o.

Funny Thing, But You Can Sing It With A Lie In Your Voice
And Before You Know, Start To Feeling Good
You Simply Got No Choice

Samsung's Screwed
They Let Google Show Them
Samsung's Screwed
By Their Lies you'll Know Them
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #124 of 153
Quote:
Originally Posted by LAKings33 View Post

This is only in the International Exynos 5410 version of the Galaxy S4, NOT the Snapdragon 600 model.

It would also be worth pointing out that even without the overclock, the Exynos 5410 still offers double the performance of the A6 inside of the iPhone 5, and both are behind the Snapdragon 600.

GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p):

Apple A6 - 373 Frames

Samsung Exynos 5410 - 794 Frames

Qualcomm Snapdragon 600 - 971 Frames

Which merely points out how useless those benchmarks are - since the iPhone is easily as fast in real life (and probably faster).
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #125 of 153
Quote:
Originally Posted by QAMF View Post

1.  It does not.  However, if you want to complain about people and benchmarks, complain about Intel.  I am saying that people are making a huge deal out of something that in the grand scheme of benchmark "doping" is small.
2. would they? That depends on how bad public opinion gets.  Even Apple, who is one of the best companies in the world when it comes to PR had to admit fault when public opinion got bad enough for a problem that was super-rare.  This would require fixing the "problem" which Samsung would most likely opt to do.  From a PR perspective.

I am not defending what Samsung did.  I am saying that people should look at he larger scope.  I am saying on a scale of "how bad was the bad thing" Samsung ranks very lower.  That is not to say what Samsung did should be defended, that is to say that many companies that do worse.

Where was Apple Insider's story when Intel and Antutu happened?  The story that talked about how x86 was going to surpass ARM for power efficiency, which does matter to what SoC Apple could possibly use. (Intel was willing to license Atom in the past, had no takers.  Apple could do custom Atom if it was faster at the given power envelope).   Something that was far worse and not fixable (at the same power envelope, I suppose you could increase the clocks of the Atom by 20% or more, but that would drastically raise battery life) https://www.google.com/search?q=antutu+Intel&oq=antutu+Intel&aqs=chrome.0.69i57j0l3j69i62l2.2090j0&sourceid=chrome&ie=UTF-8

If you dislike this kind of stuff than go after everyone.  On the scale of "how badly _____ company cheat(s/ed) benchmarks" Samsung is not very high up.  Yet people are making a huge deal of out it.  What Samsung did is wrong.  It is "less wrong" than what Intel continues to do, and Nvidia to a lesser extent than Intel.  It is just what most companies that make their own hardware do.

-QAMF

You keep going on about with irrelevant horse shit, and yes you are defending Samsung with this horse shit. 

iMac 2007, Macbook pro 2008, Mac Mini 2011
Reply
iMac 2007, Macbook pro 2008, Mac Mini 2011
Reply
post #126 of 153
This is so funny. Reminds me of the cheating Korean students in college. Yes, they were all cheating all the time.
post #127 of 153
Quote:
Originally Posted by Gatorguy View Post

I hope you're not waiting for me to suddenly become a Samsung supporter.1rolleyes.gif
No, just looking for the condescending reply, wrapped in a backhanded complement, that reveals to us - the average Joe - how we have mistakenly arrived at such a backwards view of reality. I'm actually here for humor as well as information, and your posts always seem to supply more of the former than the latter.

We've always been at war with Eastasia...

Reply

We've always been at war with Eastasia...

Reply
post #128 of 153
Quote:
Originally Posted by ankleskater View Post

They both provide historical facts and technical explanations to back up their opinions/posts. Their posts do not always go against Apple either. How balanced are your posts? How often do you provide technical insight? In fact, how much (little) do you understand the industry, either technically or from an investment perspective? 
Yes but it's the superior attitude and condescension that they always provide that warms the heart and educates the masses...kind of like what you just did. To answer your question, I know nothing about the industry, but I never proclaimed to. Also I'm not interested in the investment perspective because investment in consumer tech does not offer enough return for the volatility involved. I do much better elsewhere. I did think this forum however was more for basic information and not a trade forum, so excuse for wandering in and trying to sit at the adults table.

We've always been at war with Eastasia...

Reply

We've always been at war with Eastasia...

Reply
post #129 of 153
Quote:
Originally Posted by punkndrublic View Post

You keep going on about with irrelevant horse shit, and yes you are defending Samsung with this horse shit. 

I dislike all cheating on benchmarks, I wish people would start caring about it more.

I give this a much lower priority than I give to Intel's antutu cheat because of the nature and the %.

Samsung should not have done this.

Samsung had almost no reason to do this.

-QAMF

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply
post #130 of 153
Quote:
Originally Posted by jragosta View Post

Quote:
Originally Posted by Dick Applebaum View Post

Funny... This article reminded me of a song:

I figured I could have some fun changing a few words in the lyrics,,,

]

How's this?

NEIL DIAMOND - SONG SUNG BLUE LYRICS

Samsung's screwed
Now that they've been busted
Samsung's screwed
Cause they can't be trusted

Samsung's Lies Are Subject To The Blues Now And Then
But When You Take The Lies And Publicize
You Might Get Caught Again
You Will get Caught Again

Samsung's Screwed
Weeping like a willow
Samsung's Screwed
Hear Them Pay Their Shill-o.

Funny Thing, But You Can Sing It With A Lie In Your Voice
And Before You Know, Start To Feeling Good
You Simply Got No Choice

Samsung's Screwed
They Let Google Show Them
Samsung's Screwed
By Their Lies you'll Know Them

LOL... you are in the wrong business... You could be a lyricist or a composer...

Seriously, I wish I had the voice to record that!
"Swift generally gets you to the right way much quicker." - auxio -

"The perfect [birth]day -- A little playtime, a good poop, and a long nap." - Tomato Greeting Cards -
Reply
"Swift generally gets you to the right way much quicker." - auxio -

"The perfect [birth]day -- A little playtime, a good poop, and a long nap." - Tomato Greeting Cards -
Reply
post #131 of 153
Somebody has to be paying this guy.

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #132 of 153

I love the one and only comment on the android geek site.

 

 

Sonny Gonzalez 

My s4 bricked I followed every step. I put back stock kernel and now my device says it detected an application trying to access information and stopped loading. Now my Wi-Fi does not work. Help

 
 
  •  
post #133 of 153

^New troll inbound. Brand new account eh?

post #134 of 153
Quote:
Originally Posted by Suddenly Newton View Post

Somebody has to be paying this guy.

 

Not a bad gig, get paid for posting rubbish on the internet. But Samsung's cost/benefit analysis (amongst other things) must be screwed, as these guys aren't (and haven't been) convincing anyone.

"We're Apple. We don't wear suits. We don't even own suits."
Reply
"We're Apple. We don't wear suits. We don't even own suits."
Reply
post #135 of 153
Scumbags? Unscrupulous? Seriously??

They have a GPU run at it's highest potential during benchmarks, and reduce it to what will run any game with no problem or app. This is purely to save battery, keep heat down and so on. Who cares if it is software activated. Given that the kernel/OS is open source, this can be tracked down and changed very easily, in just a few steps. It could also be software unlocked via update. Simple.

Cyanogenmod for example, is easy to install and allows overclocking or GPU clock setting as an option, do you have that option on any iOS? No.

Really this is an incredibly minor thing, of course being blown up by AI.
post #136 of 153
Quote:
Originally Posted by phimuskapsi View Post

Scumbags? Unscrupulous? Seriously??

They have a GPU run at it's highest potential during benchmarks, and reduce it to what will run any game with no problem or app. This is purely to save battery, keep heat down and so on. Who cares if it is software activated. Given that the kernel/OS is open source, this can be tracked down and changed very easily, in just a few steps. It could also be software unlocked via update. Simple.

Cyanogenmod for example, is easy to install and allows overclocking or GPU clock setting as an option, do you have that option on any iOS? No.

Really this is an incredibly minor thing, of course being blown up by AI.

No, the issue is that the highest clock the GPU can reach with the current code being supplied to run it is 480Mhz outside of the benchmarks.

While it is 533 (532 in Anandtech's test) in the benchmarks.

Seeing as you cannot reach 533 default in any conditions except for some benchmarks not all, Samsung did not do as you claim.  iF If it was for all benchmarks, perhaps what you are saying would be right.  Oh, and if code was not found that made it run at those higher clocks for those benchmarks...

Is it being overblown by AI?  I would say yes considering larger cases than than this one that concerned low power level chips that will be in phones (and tablets) competing with Apple this year have been ignored.

Should it be swept under the rug?  Certainly not.  This article is far better than nothing, or a quick dismissal.

-QAMF

ADDENDUM: Wonder how long until Samsung's phones become like this one: https://semiaccurate.com/assets/uploads/2013/01/Sony_Core_i7_phone_prototype.jpg?

That is what a phone on steroids would look like!   At least the battery life would theoretically be good, right?


Edited by QAMF - 7/30/13 at 10:00pm

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply
post #137 of 153

product manufacturer makes its stuff run benchmarks well

 

duh

 
ai looks sadder every day

 

when the powerpc macs, which i personally loved and still have some going, were getting trashed in benchmark after benchmark, apple and fanboys whined for years that benchmarks didn't count

 

they didn't then, they don't now

 

the vast majority of computing systems are far more powerful than necessary, gamers who say they 'need' ever more fps would do better to get a life, outside of hard real time/safety critical process control and doing useful* scientific/medical/engineering stuff it really doesn't matter that much, especially not on fondlephones/slabs with tiny screens

 

* high frequency trading? no, ban it or tax transactions at 0.01 euro/dollar/pound/yen/renminbi, it's parasitic drain on the real economy with no net benefit to humanity

post #138 of 153
Quote:
Originally Posted by phimuskapsi View Post

Scumbags? Unscrupulous? Seriously??

They have a GPU run at it's highest potential during benchmarks, and reduce it to what will run any game with no problem or app. This is purely to save battery, keep heat down and so on. Who cares if it is software activated. Given that the kernel/OS is open source, this can be tracked down and changed very easily, in just a few steps. It could also be software unlocked via update. Simple.

Cyanogenmod for example, is easy to install and allows overclocking or GPU clock setting as an option, do you have that option on any iOS? No.

Really this is an incredibly minor thing, of course being blown up by AI.

 

I think you are technically correct that the whole point of Exynos Octa Big.Little design is to optimize performance/power consumption on mobile devices.  In that spirit, this benchmarkboost is all legit, but Samsung should have made it all public.  It's petty, but still unethical.

 

At the same time, Samsung never promised "twice as fast or half the price" (eg, remember iPhone's 3G marketing campaign) or failed to deliver (eg, -- *ahem*  *cough* *cough* -- Apple's Mapgate), so this is clearly overblown.

post #139 of 153
Quote:
Originally Posted by phimuskapsi View Post

Scumbags? Unscrupulous? Seriously??

They have a GPU run at it's highest potential during benchmarks, and reduce it to what will run any game with no problem or app. This is purely to save battery, keep heat down and so on. Who cares if it is software activated. Given that the kernel/OS is open source, this can be tracked down and changed very easily, in just a few steps. It could also be software unlocked via update. Simple.

Cyanogenmod for example, is easy to install and allows overclocking or GPU clock setting as an option, do you have that option on any iOS? No.

Really this is an incredibly minor thing, of course being blown up by AI.

lol

You said it mstone !
post #140 of 153
Quote:
Originally Posted by Pujones1 View Post


But people buy their stuff anyway and think they better than Apple and are somehow getting the best deal out there without paying a higher price.

 

 


That is the single reason of success of Samcrap in Germany: mentality there is interesting: As long as it is an imported product, they will look for the best price or best price/performance deal and convince themselves they made a hell of a deal. When there is german product in question, then they are full of emotions...

post #141 of 153
Quote:
Originally Posted by Bao D Nguyen View Post

This is so funny. Reminds me of the cheating Korean students in college. Yes, they were all cheating all the time.

 

Please let's not go down this road.  There are cheaters, liars, criminals in every ethnic groups. 

post #142 of 153
Who really pays attention to these benchmarks except people like Anand and the market he caters to?
post #143 of 153

That is pretty low of Samsung.

 

Intel do something like this too.  Apparently their C++ compiler  checks to see if the CPU vendor ID is genuineintel and if it isn't it generates intentionally sub-optimal code.
 

post #144 of 153
Quote:
Originally Posted by patrickwalker View Post

Who really pays attention to these benchmarks…

Unacceptable reaction.

Originally Posted by Slurpy

There's just a TINY chance that Apple will also be able to figure out payments. Oh wait, they did already… …and you’re already fucked.

 

Reply

Originally Posted by Slurpy

There's just a TINY chance that Apple will also be able to figure out payments. Oh wait, they did already… …and you’re already fucked.

 

Reply
post #145 of 153
Quote:
Originally Posted by LAKings33 View Post

This is only in the International Exynos 5410 version of the Galaxy S4, NOT the Snapdragon 600 model.

 

It would also be worth pointing out that even without the overclock, the Exynos 5410 still offers double the performance of the A6 inside of the iPhone 5, and both are behind the Snapdragon 600.

 

GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p):

 

Apple A6 - 373 Frames

 

Samsung Exynos 5410 - 794 Frames

 

Qualcomm Snapdragon 600 - 971 Frames

Is that supposed to be frames per second? If it is, who the hell cares? You can't play the games at those frame rates anyway. So who the hell cares other than someone trying to epeen brag?

post #146 of 153
Clearly fraud. A class of Galaxy S4 buyers ought to get themselves a free iPhone out of Samsung.

I wasn't damaged. I'm paying attention. But somebody must have been.
post #147 of 153
Quote:
Originally Posted by diplication View Post

I'm waiting for Gatorguy and KDarling to weigh in on this issue!  1biggrin.gif

 

Really?  Okay.  

 

It was underhanded.  It was also extremely dumb, publicity and reputation wise, for someone at Samsung to do this.  They need to send an edict from the top, telling their employees to straighten up or expect to be fired.

 

At the same time, it's hardly shocking or rare. That's why I didn't care to comment.  It's like a news article about someone getting an email from a fake Nigerian prince.  Tuning for benchmarks has gone on since benchmarks were invented.  

 

Remember the 2003 Power Mac G5 benchmark scandal?  Where Steve Jobs quoted tests from a company they hired to "prove" that the G5 outperformed Intel desktops?   It turned out they used Apple supplied tools to tweak the performance, and also changed the malloc code to work faster.

 

Graphics card makers are infamous for watching for benchmarks.

 

Various browsers often had / have code that watches for benchmarks.  Then there's the opposite problem.  Sometimes companies come up with their own benchmarks that favor themselves.  Microsoft has done that.

 

This is why benchmarks need to be changed fairly often, so that programmers can't watch for them.

 

The upshot is, I'd actually be more surprised if every OS didn't have something tweaked to look good in benchmarks, even if they don't go so far as to explicitly watch for certain apps.  That's why I tell people to never trust benchmarks.  Or emails from Nigerian princes.

post #148 of 153
Quote:
Originally Posted by jragosta View Post


Which merely points out how useless those benchmarks are - since the iPhone is easily as fast in real life (and probably faster).

Actually you fail to understand the purpose of those benchmarks and how they are applied. UI optimization is different from 3D performance, something the A6 is lacking in (it's also missing key API support for OpenGL ES 3.0, something only the Adreno 320 inside of the Snapdragon SoC has). You also fail to realise those scores are to compare SoCs on an equal playing field, hence offscreen 1080p. On screen performance is at a different resolution which depends on the device.

 

Example - native resolution performance:

 

Apple iPad 4 (A6X) - 2048x1536 - 713 Frames

 

Samsung Galaxy S4 (Exynos 5410) - 1920x1080 - 794 Frames

 

iPhone 5 (A6) - 1136x640 - 817 Frames

 

Samsung Galaxy S4 (Snapdragon 600) - 1920x1080 - 971 Frames

 

Google/LG Nexus 4 (Snapdragon S4 Pro) - 1280x720 - 1252 Frames

 

Samsung Galaxy S4 LTE-Advanced (Snapdragon 800) - 1920x1080 - 1479 Frames

 

As you can see, the iPhone 5 has a much improved performance at its native resolution allowing it to even outperform the iPad 4.


Edited by LAKings33 - 7/31/13 at 8:49am
post #149 of 153
Quote:
Originally Posted by MikeJones View Post

Is that supposed to be frames per second? If it is, who the hell cares? You can't play the games at those frame rates anyway. So who the hell cares other than someone trying to epeen brag?

No, the Fps score can be calculated from that.

 

GFXBench 2.7 T-Rex HD C24Z16 - Offscreen (1080p):

 

NVIDIA Tegra 3 - 234 Frames (4.2 Fps)

 

Apple A6 - 373 Frames (6.7 Fps)

 

Samsung Exynos 5410 - 794 Frames (14.2 Fps)

 

Qualcomm Snapdragon S4 Pro - 871 Frames (15.6 Fps)

 

Apple A6X - 971 Frames (17.3 Fps)

 

Qualcomm Snapdragon 600 - 971 Frames (17.3 Fps)

 

NVIDIA Tegra 4 - 1363 Frames (24.3 Fps)

 

Qualcomm Snapdragon 800 - 1479 Frames (26.4 Fps)

post #150 of 153
Re: the Intel benchmarking. I once consulted at Houston Instuments (they made plotters). I was hired to rewrite the code which governed how the inkjet nozzles spit out the ink droplets. In doing so, I discovered that the compiler that I was provided was optimizing some of my code away. Negating the changes they wanted. I had to turn off certain optimizations to make the desired changes. In further investigating the compiler, I discovered that the compiler optimized the code which did a memory self test away. Nothing sinister, but the compiler recognized that the code which copied the old memory data and then wrote patterns to test the memory ended up restoring the original data back in the same address without ever using the temporary patterns anywhere else...so it optimized the patterns out of the code. No one at HI recognized that the memory test did nothing because the plotter display still showed the test executing and counting down on the display depending upon the total amount of memory installed. If you examined the assembly code you wrote, everything looked ok but if you looked at the resulting machine code, you discovered the problem. I see too many programmers/engineers that don't do proper sanity checks. If it compiles with no errors and runs with no obvious errors then all is good. If it runs twice as fast as the code it replaced, then they say WOW! that optimizing compiler really works. Either inexperienced or incompetent.

Another thing, optimizing compilers have switches that allow you select which optimizations are allowed just because of things like my experience above. Also they read the cpuid in order to turn on or off the assembly code for various instructions depending on the rev and type of processor. Different processors have different capabilities depending upon their design and the cpuid is used to identifiy the differences between them. Some processors may have a bug which keeps one or more instructions from working as specified. If a compiler is aware, it can either flag instances where the instruction is used or substitute other code automatically (not my preference since it may work but be very sup-optimal for my particular purposes). Anyway what I am getting at is that the arm versus atom benchmark was probably an oops by the programmer(s) at AnTuTu. Btw, I have been retired for over 5 years now and haven't really kept up with the x86 community so I have a question - who owns AnTuTu? If Intel owns or subsidizes them I am more inclined to think this was deliberate.

I used to work for Intel and I used to work for AMD. Intel always wrote their own reference compilers but got really serious about it when when they realized that many of the commercially available compilers weren't making full use of their instruction set (C, C++ etc) with assemblers it is painfully obvious if an instruction is not fully supported but this is not so with higher level languages.Then AMD found that the Intel compilers weren't making good use of the AMD instructions so AMD startede making their own reference compilers....and so it goes. For instance in the AMD K6, AMD optimized one instuction so it executed in 4 clock cycles, the Intel Pentium at the time took iirc 15 clock cycles. Sorry I no longer remember the opcode involved but it meant that unless that opcode was absolutely necessary, the Intel compiler substituted a two opcode instruction in its place which took less than 15 clock cycles and it did the same thing on the AMD processor which made the AMD compiled code take longer than necesary. So the compiler swwitches alone can make one processor look better than another or vice-versa. I can't really reasonably comment on arm vs. atom. The soc chips I have worked with are all older, 4 bit word/12 to 16 bit address space plus some of the 8 to 16 bit data/16 to 24 bit address space x86 and 68x based socs. My real expertise has been on the x86 space from the 8088 through some of the early Xeons and the K5 through the Opterons and Athlons. The last chip I had anything to do with was the bulldozer core at AMD and I retired before that chip was released.

So, the atom vs arm benchmark may have been deliberate but I doubt it from the way things seem to have worked out. On the other hand there is no doubt that Samsung was gaming the benchmarks. That's my two cents.
post #151 of 153

GSMArena reported Samsung's response to this allegation.  http://blog.gsmarena.com/samsung-responds-to-benchmark-cheating-allegations/

 

That is;

 

Under ordinary conditions, the Galaxy S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the Galaxy S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

We remain committed to providing our customers with the best possible user experience.

post #152 of 153
Headline of the year!
Apple has no competition. Every commercial product which competes directly with an Apple product gives the distinct impression that, Where it is original, it is not good, and where it is good, it...
Reply
Apple has no competition. Every commercial product which competes directly with an Apple product gives the distinct impression that, Where it is original, it is not good, and where it is good, it...
Reply
post #153 of 153
Quote:
Originally Posted by Chick View Post

Re: the Intel benchmarking. I once consulted at Houston Instuments (they made plotters). I was hired to rewrite the code which governed how the inkjet nozzles spit out the ink droplets. In doing so, I discovered that the compiler that I was provided was optimizing some of my code away. Negating the changes they wanted. I had to turn off certain optimizations to make the desired changes. In further investigating the compiler, I discovered that the compiler optimized the code which did a memory self test away. Nothing sinister, but the compiler recognized that the code which copied the old memory data and then wrote patterns to test the memory ended up restoring the original data back in the same address without ever using the temporary patterns anywhere else...so it optimized the patterns out of the code. No one at HI recognized that the memory test did nothing because the plotter display still showed the test executing and counting down on the display depending upon the total amount of memory installed. If you examined the assembly code you wrote, everything looked ok but if you looked at the resulting machine code, you discovered the problem. I see too many programmers/engineers that don't do proper sanity checks. If it compiles with no errors and runs with no obvious errors then all is good. If it runs twice as fast as the code it replaced, then they say WOW! that optimizing compiler really works. Either inexperienced or incompetent.

Another thing, optimizing compilers have switches that allow you select which optimizations are allowed just because of things like my experience above. Also they read the cpuid in order to turn on or off the assembly code for various instructions depending on the rev and type of processor. Different processors have different capabilities depending upon their design and the cpuid is used to identifiy the differences between them. Some processors may have a bug which keeps one or more instructions from working as specified. If a compiler is aware, it can either flag instances where the instruction is used or substitute other code automatically (not my preference since it may work but be very sup-optimal for my particular purposes). Anyway what I am getting at is that the arm versus atom benchmark was probably an oops by the programmer(s) at AnTuTu. Btw, I have been retired for over 5 years now and haven't really kept up with the x86 community so I have a question - who owns AnTuTu? If Intel owns or subsidizes them I am more inclined to think this was deliberate.

I used to work for Intel and I used to work for AMD. Intel always wrote their own reference compilers but got really serious about it when when they realized that many of the commercially available compilers weren't making full use of their instruction set (C, C++ etc) with assemblers it is painfully obvious if an instruction is not fully supported but this is not so with higher level languages.Then AMD found that the Intel compilers weren't making good use of the AMD instructions so AMD startede making their own reference compilers....and so it goes. For instance in the AMD K6, AMD optimized one instuction so it executed in 4 clock cycles, the Intel Pentium at the time took iirc 15 clock cycles. Sorry I no longer remember the opcode involved but it meant that unless that opcode was absolutely necessary, the Intel compiler substituted a two opcode instruction in its place which took less than 15 clock cycles and it did the same thing on the AMD processor which made the AMD compiled code take longer than necesary. So the compiler swwitches alone can make one processor look better than another or vice-versa. I can't really reasonably comment on arm vs. atom. The soc chips I have worked with are all older, 4 bit word/12 to 16 bit address space plus some of the 8 to 16 bit data/16 to 24 bit address space x86 and 68x based socs. My real expertise has been on the x86 space from the 8088 through some of the early Xeons and the K5 through the Opterons and Athlons. The last chip I had anything to do with was the bulldozer core at AMD and I retired before that chip was released.

So, the atom vs arm benchmark may have been deliberate but I doubt it from the way things seem to have worked out. On the other hand there is no doubt that Samsung was gaming the benchmarks. That's my two cents.

the Antutu compiler changed part of the code to all 0's

 

 

Quote: Exophase (http://forums.anandtech.com/showthread.php?t=2330027)
What it's doing is, where possible, setting entire 32 bit runs to 0 or 1. The lines at f64c3 and f64c6 are critical. It's replacing 32 iterations of the ARM loop above with those two instructions. Needless to say, it's dozens of times faster doing it this way.

This is what we call breaking the benchmark. Where the compiler applies some logic that makes the benchmark much faster by doing a set of operations that the benchmark identifies as correct (if it even checks) but are not performing the intended function of the benchmark. Classic examples include omitting code entirely if the results are never read, or performing a complex computation at compile-time instead of run time if the inputs can determined to be constant (then just reporting the results).

In this case I'm sure Intel could claim that they're performing a legitimate optimization. Frankly, I doubt it; this kind of optimization would be difficult to recognize and apply in generic code. It'd also be for little benefit, because I've never seen someone use code like this to set or clear huge sets of bits. That part is kind of the catch, because this optimization would make the code slower if the run lengths weren't sufficiently large.  In nbench's case they are, but there's no way the compiler could have known that on its own.

What's more, this optimization wasn't present in ICC until a recent release. Somehow I don't think that they just now discovered it has general purpose value. More likely case is that they discovered is they could manipulate AnTuTu's scores.

How is this utilizing more of the Intel CPU?  This is blatantly cheating the purpose of the benchmark.  This does not make the Intel CPU run the benchmark faster thanks to special functions, it makes the benchmark easier to run.

Much, much worse than what the S4 does, especially in light of Samsung's response, which, while not great, did show it wasn't just in some benchmarks (Barely)


-QAMF

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply

Active on S}A forums.  S|A student level subscriber.  Don't claim to know what is in the articles.

Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Galaxy S 4 on steroids: Samsung caught doping in benchmarks