or Connect
AppleInsider › Forums › Mobile › iPhone › Samsung again caught doping benchmarks for Galaxy Note 3
New Posts  All Forums:Forum Nav:

Samsung again caught doping benchmarks for Galaxy Note 3 - Page 3

post #81 of 103
High Tech Cheating
post #82 of 103
Quote:
Originally Posted by EricTheHalfBee View Post
 

 

Now I remember why I added him to my block list. Although when someone quotes him I'm stuck reading the stupidity.

 

It's so funny how the forums are exploding everywhere with Samsung apologists. You can tell it really bothers them that Apple has made such a great SoC in the A7.

 

For close to 30 years people have been discussing processor efficiency. It's long been agreed that a processor that can do the same work at a lower clock has the superior architecture. Likewise with a processor that can do the same work with fewer cores. Or a processor that does it while consuming less power.

 

But now we have a serious problem in the universe. Apple, a useless company that doesn't innovate, has created a processor that does all three tremendously well. This has caused scores of people (like Frood) to simply throw out 30 years of accepted facts regarding what constitutes good processor design simply because it's made by a company they dislike.

 

This goes far beyond the usual level of "pathetic".

 

First of all, I am not a Samsung apologist.  And that's fine that you blocked me for whatever the reason.  

 

This limiting CPU speed for some of its applications in Android are not new as Ars notified.  So this is a hardly news.  

 

But calling  'benchmark booster', 'faking performance', 'benchmark shenanigans' to one of Android manuracturers and accusing them as 'cheater' because they tested benchmarks at the CPU speed that was advertised for is pure evil journalism to me.

post #83 of 103
Good job Samsung, you and your products are just one big lie!
post #84 of 103
Quote:
Originally Posted by hjb View Post
 

But calling  'benchmark booster', 'faking performance', 'benchmark shenanigans' to one of Android manuracturers and accusing them as 'cheater' because they tested benchmarks at the CPU speed that was advertised for is pure evil journalism to me.

Just so you are clear, I will remind you that the hardware, you so oh hail as "advertised" was advertised to deliver yay amount of performance, while delivering yay amount of battery backup. In Note 3's case, they promise certain amount of battery backup under normal usage conditions. These usage conditions employ the CPU to run at a power efficient state (which is normally what you use). It is just to call it "faking" because by putting the CPU to a performance state, it is delivering its full performance, yes, but at the cost of reduced battery life.

For the result not be a fake, either:

​1. Samsung use the normal CPU state policy across all apps

​2. Or, Samsung use the power policy and openly state a much reduced battery life.

​The fact that Samsung uses power policy just for benchmarks (to improve results), and still claim that the CPU runs at normal power (which is given by way of official battery backup specifications), is pure unadulterated fraud.

post #85 of 103
Quote:
Originally Posted by crysisftw View Post
 

Just so you are clear, I will remind you that the hardware, you so oh hail as "advertised" was advertised to deliver yay amount of performance, while delivering yay amount of battery backup. In Note 3's case, they promise certain amount of battery backup under normal usage conditions. These usage conditions employ the CPU to run at a power efficient state (which is normally what you use). It is just to call it "faking" because by putting the CPU to a performance state, it is delivering its full performance, yes, but at the cost of reduced battery life.

For the result not be a fake, either:

​1. Samsung use the normal CPU state policy across all apps

​2. Or, Samsung use the power policy and openly state a much reduced battery life.

​The fact that Samsung uses power policy just for benchmarks (to improve results), and still claim that the CPU runs at normal power (which is given by way of official battery backup specifications), is pure unadulterated fraud.

 

I will be calling them 'cheater' only if they have overclocked its CPU when performing the benchmark tests.  But they did not as long as I know.

 

And any manufacturer can limit its CPU or any of its components to optimize their user experience for normal users and of course all have been doing it ages.  What is wrong with it?  

 

Car manufactures advertise how fast they could go like 150miles per hour as a benchmark, but would anyone be complaining why they do not set benchmark 60miles an hour?

 

Benchmark is just a benchmark.


Edited by hjb - 10/1/13 at 9:50pm
post #86 of 103
Quote:
Originally Posted by hjb View Post
 

 

First of all, I am not a Samsung apologist.  And that's fine that you blocked me for whatever the reason.  

 

This limiting CPU speed for some of its applications in Android are not new as Ars notified.  So this is a hardly news.  

 

But calling  'benchmark booster', 'faking performance', 'benchmark shenanigans' to one of Android manuracturers and accusing them as 'cheater' because they tested benchmarks at the CPU speed that was advertised for is pure evil journalism to me.

 

The LG G2 having absolutely the same CPU seems to lag behind in those performance tests. So, it seems that only Samsung is capable of "limiting CPU speeds"?

 

While scaling down the CPU is obviously an industry practice, boosting it is not. Intel does that, calling it turbo boost, but it is available for each and every application running on your computer, no matter if it is a Windows, Linux or Mac OS X box.

 

Samsung does that specifically and only for performance tests. Those applications don't do any real work, but measure CPU capabilities. So, yes, that is the exact definition of cheating. It is a fake performance, because it is not achievable in actual applications. And it is "benchmark shenanigans", since Samsung did not explicitly state they have special configuration for benchmark apps.

post #87 of 103
Quote:
Originally Posted by capasicum View Post
 

 

1. 64-bit CPUs do not have advantage over 32-bit CPUs when running 32-bit tests/benchmarks as the software does not take advantage of the additional registers and their greater size. ARM Cortex A15 already has a 64-bit data path between RAM and CPU cache.

 

2. Snapdragon is running at a 77% higher frequency, meaning more than twice the power needed (non-linear increase).

 

3. Snapdragon has twice the cores, meaning twice the consumption when all cores work (as is the case with benchmarks). That's why ARM created the big.LITTLE architecture. Samsung's implementation, however, is awful.

 

So, for close to 5 times the power consumption you get mind-blowing 3% better performance scores.

 

4. Benchmarks usually scale extremely well (close to perfect) on multi-core platforms. That is never the case with real-world applications, except for some cases that are usually run on supercomputers, or on GPUs.

 

So, it seems to me that the iPhone 5s has the most powerful mobile CPU. Apple chose not to over-do it with 4 cores, nor boost the frequency. Because A7's  purpose is to power a phone, not a data center.

 

Well...the above post ended all the argument.  Good job!

post #88 of 103
Quote:
Originally Posted by hjb View Post
 

 

I will be calling them 'cheater' only if they have overclocked its CPU when performing the benchmark tests.  But they did not as long as I know.

 

And any manufacturer can limit its CPU or any of its components to optimize their user experience for normal users and of course all have been doing it ages.  What is wrong with it?  

 

Car manufactures advertise how fast they could go like 150miles per hour as a benchmark, but would anyone be complaining why they do not set benchmark 60miles an hour?

 

Benchmark is just a benchmark.

 

 

I still don't think that you get what I am saying. Either Samsung should use the same state policy across all apps (to deliver the advertised performance with the advertised battery life), or use the power policy that they use in benchmarks, in all apps and openly admit that the battery life, CPU life have significantly decreased and there is a lot of heat generation. The fact that Samsung uses the power policy only for benchmarks is the act of window-dressing and that alone is misrepresentation.
 

Didn't you see the post? When they changed the name of the app, the performance decrease was 35% at an average, or put simply, the real performance of Note 3 is actually 35% less than what it has you believe in benchmarks. Isn't that misrepresentation?

 

 

post #89 of 103
Crysisftw.


Why manufacturer should use the same power policy across the board? It is up to them for optimisation.

What Sammy has done wrong again?
post #90 of 103
Quote:
Originally Posted by hjb View Post

Crysisftw.


Why manufacturer should use the same power policy across the board? It is up to them for optimisation.

What Sammy has done wrong again?

 

See this comment by Anand of Anandtech:

http://anandtech.com/comments/7376/samsung-galaxy-note-3-review/331108

 

Benchmark boosting is widespread among Android OEMs, cut throat market I suppose, reminds me a little of the dynamic contrast ratio ratings of monitors, we end up seeing ridiculous numbers and it becomes meaningless. It already is kind of meaning less between OS platforms, if not just for the reason that if say you are loyal to Android, why does it matter to you what a Windows phone or iPhone scores?

post #91 of 103
Quote:
Originally Posted by murman View Post

See this comment by Anand of Anandtech:
http://anandtech.com/comments/7376/samsung-galaxy-note-3-review/331108

Benchmark boosting is widespread among Android OEMs, cut throat market I suppose, reminds me a little of the dynamic contrast ratio ratings of monitors, we end up seeing ridiculous numbers and it becomes meaningless. It already is kind of meaning less between OS platforms, if not just for the reason that if say you are loyal to Android, why does it matter to you what a Windows phone or iPhone scores?

Did you note this additional comment he made?
"Google Experience (Nexus) devices aren't affected" which shows it's being done intentionally by some of the others. Google plays it straight with Nexus models, as they should anyway especially since they're used to premier new OS versions.
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #92 of 103
Quote:
Originally Posted by hjb View Post

Why manufacturer should use the same power policy across the board? It is up to them for optimisation.

What Sammy has done wrong again?

 

Can any app opt into the aggressive power policy that Samsung uses for those benchmarks? If so, why does Samsung hardcode those benchmarks in a special whitelist? If not, wouldn't the benchmark scores lead consumers to wrongly conclude that the Note 3 can run real-world apps significantly faster than other similarly equipped devices?


Edited by d4NjvRzf - 10/2/13 at 3:36am
post #93 of 103
First, Note 3 runs benchmarks at 2.3Ghz that SD800 is advertised for. So, there is no such thing like "benchmark boosting". ie. Note 3 does not overclock when performing benchmark tests.

I am not royal to Android. My family still enjoy more than 2 years old iPad2 and recommends iPad to anyone, although I prefer Android for smartphones over iOS.

As you might see from above, I do not have any interest what a Windows phone or iPhone scores, for now of course.
post #94 of 103
Quote:
Originally Posted by hjb View Post

First, Note 3 runs benchmarks at 2.3Ghz that SD800 is advertised for. So, there is no such thing like "benchmark boosting". ie. Note 3 does not overclock when performing benchmark tests. I am not royal to Android. My family still enjoy more than 2 years old iPad2 and recommends iPad to anyone, although I prefer Android for smartphones over iOS. As you might see from above, I do not have any interest what a Windows phone or iPhone scores, for now of course.

That was to murman.
post #95 of 103

Cheating in more ways than one

post #96 of 103
2013-09-30-20.391.jpg

Yeah, nothing whatsoever to see here.
"We're Apple. We don't wear suits. We don't even own suits."
Reply
"We're Apple. We don't wear suits. We don't even own suits."
Reply
post #97 of 103

Samsung: "This is how we are wired…" ;-)

post #98 of 103
Quote:
Originally Posted by hjb View Post

First of all, I am not a Samsung apologist.  And that's fine that you blocked me for whatever the reason.  

This limiting CPU speed for some of its applications in Android are not new as Ars notified.  So this is a hardly news.  

But calling  
'benchmark booster', 'faking performance', 'benchmark shenanigans' to one of Android manuracturers and accusing them as 'cheater' because they tested benchmarks at the CPU speed that was advertised for is pure evil journalism to me.

I beg to differ. You are a Samsung apologist.
Quote:
Originally Posted by hjb View Post

Crysisftw.


Why manufacturer should use the same power policy across the board? It is up to them for optimisation.

What Sammy has done wrong again?

Optimization is one thing. 'Optimization' would involve figuring out what the app does and then assigning the appropriate conditions for that app. But when the same app is 'optimized' differently by merely changing it's name, that's not optimization. It's artificially giving known benchmark apps an advantage that other apps don't have.

The purpose of benchmarks is to give you an easily measured performance metric that allows you to compare one device to another. Ideally, if device A is 30% faster in real life, then it should be 30% faster on the benchmark. With this type of gamesmanship, the benchmark loses all value. It is 'optimized' differently than the apps that people are using and is therefore useless as an indicator of which device is faster.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #99 of 103
Quote:
Originally Posted by Gatorguy View Post


Did you note this additional comment he made?
"Google Experience (Nexus) devices aren't affected" which shows it's being done intentionally by some of the others. Google plays it straight with Nexus models, as they should anyway especially since they're used to premier new OS versions.

 

I didn't read every comment, but hey he wrote them, not me. And gooooood for you if you own a Nexus, have a cookie.

post #100 of 103

Those who are saying Samsung did not cheat are total idiots.  Just ignore the idiots.

 

Samsung again proves they are a company with ZERO ethics.

No surprise since their CEO is a three time convicted FELON.

post #101 of 103
Quote:
Originally Posted by jragosta View Post

The purpose of benchmarks is to give you an easily measured performance metric that allows you to compare one device to another. Ideally, if device A is 30% faster in real life, then it should be 30% faster on the benchmark. With this type of gamesmanship, the benchmark loses all value. It is 'optimized' differently than the apps that people are using and is therefore useless as an indicator of which device is faster.

^This.

post #102 of 103

Why the hell are we comparing a small tablet vs a smart phone?

 

What next?  Comparing an iMac to the Note3?  The Note3 is literally 2x bigger than the iPhone5S.

post #103 of 103
Quote:
Originally Posted by murman View Post

I didn't read every comment, but hey he wrote them, not me. And gooooood for you if you own a Nexus, have a cookie.

Has nothing to do with which device you use. It speaks to how reliable the benchmarking results are. If it's a Nexus device they're not gaming the tests. If it's Samsung it's likely they are.

EDIT: I see DED has posted an article here at AI pointing out the exact same thing. Daniel can haz cookies too?
http://appleinsider.com/articles/13/10/02/asus-htc-lg-and-other-android-licensees-join-samsung-in-faking-test-results
Edited by Gatorguy - 10/2/13 at 11:12am
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Samsung again caught doping benchmarks for Galaxy Note 3