or Connect
AppleInsider › Forums › Mobile › iPhone › Asus, HTC, LG and other Android licensees join Samsung in faking test results
New Posts  All Forums:Forum Nav:

Asus, HTC, LG and other Android licensees join Samsung in faking test results

post #1 of 130
Thread Starter 
Cheating test scores and misrepresenting device capabilities are rampant at Samsung, but are also being used by virtually every other Android licensee apart from Google itself, research shows.

Samsung cheat.jar


Samsung has been shamed twice this year by confirmed reports of its intentional falsification of performance benchmarks involving both of its flagship products: the Galaxy S4 and Note 3.

According to testing by Anand Lal Shimpi and Brian Klug of AnandTech, benchmark cheats are not limited to Samsung, but rather pandemic among Android licensees.

"With the exception of Apple and Motorola," the site observed, "literally every single OEM we?ve worked with ships (or has shipped) at least one device" that similarly fudges benchmarks.

Even the cheating on Android is fragmented



However, since each Android licensee cheats in different ways on different types of benchmarks, it's difficult to benchmark the benchmark cheating. It's also hard to convince them to stop, authors stated.

"Virtually all Android vendors appear to keep their own lists of applications that matter and need optimizing," the report observed. "The lists grow/change over time, and they don?t all overlap. With these types of situations it?s almost impossible to get any one vendor to be the first to stop. The only hope resides in those who don?t partake today, and of course with the rest of the ecosystem."

The lone holdout to Android's rampant cheating is Google and its Motorola subsidiary and Nexus brand. However, those products collectively make up only a tiny fraction of Android sales, and are intended to serve as a guiding reference model to other Android licensees.

Specifically, Google's "pure Android" products intend to demonstrate to mass Android manufacturers how to deliver clean, updated products that take advantage of features in the latest version of the OS, enable uses to get timely updates after purchase and direct users' attention and revenue exclusively to Google.

Android licensees haven't followed Google's lead in any of those respects. Instead, Chinese and Korean hardware makers have done the same thing they did to Microsoft Windows in the PC market: race to the bottom to deliver super cheap products that cheat on specs, substitute substandard components, skimp on build quality and direct attention to their own layers of junkware, ads and subscription plans.
post #2 of 130
Par for the course. These cats are scared, coming out of denial painfully aware that iOS's got their number and is many places to the left of their dismal trail.

Hootenanny on the horizon and Androiders not invited.

When I find time to rewrite the laws of Physics, there'll Finally be some changes made round here!

I am not crazy! Three out of five court appointed psychiatrists said so.

Reply

When I find time to rewrite the laws of Physics, there'll Finally be some changes made round here!

I am not crazy! Three out of five court appointed psychiatrists said so.

Reply
post #3 of 130
Double post yet only submitted once. Sorry.

Addendum.

But since I yam here already: kudos to Google (spit) for at least trying to set some standards for its fellow gangsters. Samsung no learny, no copy. 1biggrin.gif
Edited by mhikl - 10/2/13 at 11:28am

When I find time to rewrite the laws of Physics, there'll Finally be some changes made round here!

I am not crazy! Three out of five court appointed psychiatrists said so.

Reply

When I find time to rewrite the laws of Physics, there'll Finally be some changes made round here!

I am not crazy! Three out of five court appointed psychiatrists said so.

Reply
post #4 of 130
HAHAHAHAHAHAHAHA.
post #5 of 130
So THAT'S why the Samsung shills are so defensive: cheating on benchmarks is normal for Android phones. 1biggrin.gif

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #6 of 130

So, if Google isn't even getting ad revenue for many Android devices, why keep Android open and basically free?

post #7 of 130
Notice the similarity? American companies such as Apple and Google have a higher ethical standard and Asian companies lie and cheat regularly. Different cultures.
post #8 of 130
Quote:
Originally Posted by sambira View Post

So, if Google isn't even getting ad revenue for many Android devices, why keep Android open and basically free?

They aren't getting ad revenue from users of Google Android devices?
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #9 of 130

These fake and fraudulent test scores don't affect me of course, as I would never, ever be caught dead with one of those Android phones, but in some small sort of way, I do feel slightly sorry for all of the pathetic Fandroids out there.

 

All of the little things that Fandroids think are important are totally meaningless, and they are even more meaningless now than before. Who gives a crap about your specs or your benchmarks? Your benchmarks are a fraud and so are you, little boy. Now, go take your laggy, malware infested, 8 GB RAM, 6" octo-core fraud of a phone between your legs and hurry up home to mommy. Scoot.

 

These fake results and deceptive practices also highlight the sorry state of Android in general. Many Android vendors have no shame at all in their blatant copying, so I can't say that I'm all that surprised that they would stoop to these levels. 

post #10 of 130
i bet you wont wsj taking about this or cnbc
post #11 of 130
Originally Posted by jpmia View Post
i bet you wont wsj taking about this or cnbc

 

I dunno… since it’s more than one company, they can slip in “Android is lying to customers” somewhere.

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply
post #12 of 130
Quote:
Originally Posted by Jack Baker View Post

Notice the similarity? American companies such as Apple and Google have a higher ethical standard and Asian companies lie and cheat regularly. Different cultures.

Not nice JB. 1oyvey.gif

Look at big pharma. Scruple missing in the poor ethics race. Big corps anywhere do chase the money, leaving their morals in the outhouse.

When I find time to rewrite the laws of Physics, there'll Finally be some changes made round here!

I am not crazy! Three out of five court appointed psychiatrists said so.

Reply

When I find time to rewrite the laws of Physics, there'll Finally be some changes made round here!

I am not crazy! Three out of five court appointed psychiatrists said so.

Reply
post #13 of 130

What's pathetic behavior are the fandroids in other threads essentially spinning the cheating the Android makers are doing.  One that is beginning to show a lot of wear-and-tear is the "They are not cheating, they are simply underclocking all the normal apps to extend battery life and to prevent the CPU from running hot.  They only let benchmark apps have full resources" nonsense.

Bunch of whiny, pathetic, hypocrites.  I'll say it again like I do with everything else these Fandroids let slide.  Had Apple done anything even remotely similar, it would be on every single news outlet, and these same basement-dwelling a$$hat fandroids would be screaming class-action lawsuits against Apple.

Vermin they are.

post #14 of 130
Quote:
Originally Posted by sambira View Post

So, if Google isn't even getting ad revenue for many Android devices, why keep Android open and basically free?

It becomes more influential the more it is used, and Google along with it.

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #15 of 130
>> Originally Posted by jpmia
>> i bet you wont wsj taking about this or cnbc

> I dunno… since it’s more than one company, they can slip in “Android is lying to customers” somewhere.


They'd have to slip in that "Apple is lying to customers" before it's big "news".
post #16 of 130
Quote:
Originally Posted by AppleInsider View Post

Even the cheating on Android is fragmented


Best line of the whole article.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #17 of 130
Quote:

Originally Posted by sflocal View Post

 

Vermin they are.

 

Sir, you are too kind.

post #18 of 130
Quote:
Originally Posted by Apple ][ View Post
 

 

Sir, you are too kind.

 

Yes.. you are absolutely correct.  I apologize in earnest to all the vermins I offended by including Fandroids in their group.  What's the word for a full hole from retired outhouse?  It's full of sh!t like they are right?

post #19 of 130
Whatever.
How many people actually BUY a phone based on benchmarks?
One or three?
post #20 of 130
I think Anand Lal Shimpi and Brian Klug of AnandTech should be linked.
post #21 of 130
Quote:
Originally Posted by mhikl View Post

Par for the course. These cats are scared, coming out of denial painfully aware that iOS's got their number and is many places to the left of their dismal trail.

Hootenanny on the horizon and Androiders not invited.

 

I partially agree, and partially think they're more trying to cheat against each other than against Apple. (Given how the Android market is so focused on specs, soundbites and bullet points, to the detriment of actually delivering a good experience.)

 

But I 100% think your metaphors are rare poetry!

post #22 of 130

Right now, the tech pundit sites are furiously thinking how they might be able to blame Apple for this.

post #23 of 130

The doping mentality of tour de france bikers has infected nearly all major android OEMs.


Edited by d4NjvRzf - 10/2/13 at 1:13pm
post #24 of 130
Originally Posted by markbyrn View Post
Right now, the tech pundit sites are furiously thinking how they might be able to blame Apple for this.

 

“If Apple hadn’t created a phone that was faster than all of ours, we wouldn’t have had to lie about our benchmarks!”

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply

Originally Posted by asdasd

This is Appleinsider. It's all there for you but we can't do it for you.
Reply
post #25 of 130
Quote:
Originally Posted by davebarnes View Post

Whatever.
How many people actually BUY a phone based on benchmarks?
One or three?

The people whose self-esteem depends on it.

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #26 of 130
Quote:
Originally Posted by davebarnes View Post

Whatever.
How many people actually BUY a phone based on benchmarks?
One or three?

I'd say it's quite low, but I'd also say the number of buyers who would buy because of artificial benchmarks over usability are higher than the the number of people that wouldn't buy because of doping their chips to perform better when certain tests are noticed. Because of that I would doubt there will be enough bad press generated that would force them to stop doing it.
post #27 of 130
Quote:
Originally Posted by davebarnes View Post

Whatever.
How many people actually BUY a phone based on benchmarks?
One or three?

 

 

Probably more than you think. And it doesn't matter how many do, it's still lying.

 

It's like bragging about the engine in your Viper and then raising the hood to find a weedeater engine and an mp3 player playing sounds of a muscle car.

post #28 of 130
Quote:
Originally Posted by Suddenly Newton View Post


The people whose self-esteem depends on it.

 

You are funny.

 

 

 

I doubt the mainstream media will care about this.

So cheating does pay.

post #29 of 130
A few years ago NVidia and ATI where doing the same thing I remember.

And how do we know this is not also happening on iOS ? iOS (non-jailbreak) lacks the tools to detect these kind of things.

Edit : Found my answer in the comments from Anand :

"We have some internal tools to poke around at frequency, A7 appears to behave honestly. I explored this issue with engineers at a couple of different companies who were also interested in finding out the answer. Finally, much of what I used to characterize the A6/A7 aren't publicly available, so there's no gaming there.

The bigger concern is how much optimization for the web browsing tests exists in Browser.apk, Chrome and Mobile Safari. Like I said in the article, this is going to get a lot more difficult to detect."
post #30 of 130
Anand has been unusually vocal in the comments section of his article: http://www.anandtech.com/comments/7384/state-of-cheating-in-android-benchmarks

Anand Lal Shimpi - Wednesday, October 02, 2013

"Our plan is to just switch to privately named tests, both old and new. If the OEMs don't know what we're running, they can't optimize for it.

We do something similar on the SSD side, and honestly it's where I'd like to end up, where we have benchmarks that even the OEMs don't have. It's a taller (and more expensive) order for mobile but something I'm still trying to pursue."
post #31 of 130
I don't understand all that; some of my friends have 'droids and they seem slow to me and probably the slowest part of the equation is the user - man it takes them forever to complete a task and not one person knows now to import a vcard or send one.
post #32 of 130
Quote:
Originally Posted by davebarnes View Post

Whatever.
How many people actually BUY a phone based on benchmarks?
One or three?

 

I wonder how many trolls (not referring to you, please) troll about benchmarks?

post #33 of 130
Quote:
Originally Posted by Jack Baker View Post

Notice the similarity? American companies such as Apple and Google have a higher ethical standard and Asian companies lie and cheat regularly. Different cultures.

 

Please don't make this a racist topic.  Ever heard of Enron, Bear Sterns, Goldman Sacks, AIG, and Madoff Investments?

post #34 of 130
Quote:
Originally Posted by mhikl View Post

Par for the course. These cats are scared, coming out of denial painfully aware that iOS's got their number and is many places to the left of their dismal trail.

Hootenanny on the horizon and Androiders not invited.

A hootenanny like this?
"I got the answer by talking in my brain and I agreed of the answer my brain got" a 7 yr old explaining his math HW
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"I got the answer by talking in my brain and I agreed of the answer my brain got" a 7 yr old explaining his math HW
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
post #35 of 130
Ooh can anyone say ""Class Action Suit? Or does their "sales" not add up to enough real people to form a "class"?
post #36 of 130

These tests are stupid.  The Galaxy S4 suppose to be blazing fast.  But in the real world it lags big time.

 

From early reports it looks like even the Note3 lags! lol. losers

http://forum.xda-developers.com/showthread.php?t=2453680&page=4

http://forum.xda-developers.com/showthread.php?t=2438208

 

I don't care how fast your cpu is, if your software sucks you suck.

 

Micro-stutter: when the UI momentarily and abruptly stops during a scrolling operation (even if it's only for a couple of milliseconds). Micro-stutter interrupts what would otherwise be a smooth and fluid scrolling experience.

What causes micro-stutters? The simple answer is "jank": Problematic blocking of a software application's user interface due to slow operations. In order to maintain smoothness and fluidity in a UI, it is now generally acknowledge that a consistent 60 frames per second is needed. What this means is you have to complete all computations for everything that needs to be rendered in a single frame within 16.6 milliseconds. Failure to do so will result in "jank" (also referred to as "micro-stutter", or "frame-skip").


Firstly, I suspect (again, I'm not an Android expert) that since Android is a virtualized environment (that uses the DVM, or Dalvik Virtual Machine), it struggles to perform computations fast enough in order to complete computations within the required 16.6 milliseconds. This is why smoothness on Android devices is always second rate compared to platforms that run code natively. No matter what arguments are put forward, "native" always performs better that "virtualized".

Secondly, it looks like there's something fundamentally wrong with Android's architecture when it comes to the touch paradigm (which is now standard on just about every smartphone platform). The UI should be running in it's own non-blocking thread. No network or disk/storage operations should interfere with whatever the UI is doing (including UI inputs etc). This is not the case on Android - you can see the UI stutter wherever "lazy loading" is implemented (one example: the Play Store - stutter occurs when network operation is executed).

Thirdly, Android doesn't appear to use the GPU for screen rendering throughout the platform. From what I can remember, developers can set this within an applications manifest.xml file. I have no idea why any architect would choose to use the CPU rather than the GPU for rendering operations. In any event, you can see the difference in some applications by going into the Developer Options and turning on "Force GPU rendering". and then repeating the steps for testing the Engadget app mentioned in one of my previous posts in this thread. Scrolling is much, much smoother, but janking (micro-stutters) still occur.


Edited by sog35 - 10/2/13 at 12:56pm
post #37 of 130

Why am I not surprised, all along Android has been built on lying.

 

"We didn't steal Java."

 

"We didn't copy Apple."

 

etc, etc , etc.

A problem occurred with this webpage so it was reloaded.A problem occurred with this webpage so it was reloaded.A problem occurred with this webpage so it was reloaded.A problem occurred with this...
Reply
A problem occurred with this webpage so it was reloaded.A problem occurred with this webpage so it was reloaded.A problem occurred with this webpage so it was reloaded.A problem occurred with this...
Reply
post #38 of 130
Quote:
Originally Posted by hill60 View Post

Why am I not surprised, all along Android has been built on lying.

"We didn't steal Java."

"We didn't copy Apple."

etc, etc , etc.

Note that Google (and Motorola and Nvidia) are the only ones on the list of tested vendors that aren't doping their chips for benchmark tests.
post #39 of 130

Absolutely love the title of the first table, "I Can't Believe I Have to Make This Table"

 

LOL

Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
post #40 of 130
Quote:
Originally Posted by davebarnes View Post

Whatever.
How many people actually BUY a phone based on benchmarks?
One or three?

Enough to make Samsung, Asus, HTC, LG, and others cheat on the benchmarks.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
  • Asus, HTC, LG and other Android licensees join Samsung in faking test results
AppleInsider › Forums › Mobile › iPhone › Asus, HTC, LG and other Android licensees join Samsung in faking test results