or Connect
AppleInsider › Forums › Mobile › iPhone › Samsung Galaxy Note, HTC One caught cheating in benchmarks again
New Posts  All Forums:Forum Nav:

Samsung Galaxy Note, HTC One caught cheating in benchmarks again

post #1 of 165
Thread Starter 
Futuremark has delisted the scores of Samsung and HTC products from its public benchmark listings over new allegations of performance doping.



According to a report by Tony Smith of the Register, Futuremark accused the two companies of rigging results in its 3DMark GPU benchmarking app for Android.

The company removed the scores of Samsung's Galaxy Note 10.1 tablet and Note III phablet as well as HTC's One and One Mini smartphones, dropping the models to the bottom of its listings, below devices with a score of zero (above).

Futuremark President Oliver Baltuch told the site, "People rely on Futuremark benchmarks to produce accurate and unbiased results. That's why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software.

Baltuch explained, "in simple terms, a device must run our benchmarks without modification as if they were any other application."

Futuremark's rules stated "the platform may not detect the launch of the benchmark executable" and that devices "must not alter, replace or override any parameters or parts of the test, nor modify the usual functioning of the platform based on the detection of the benchmark."

The leading device in Futuremark's listings remains the Samsung Galaxy Round, version of the Note III with a curved body and screen and lacking a stylus. The low volume product is offered in Korea for close to $1000, essentially as a technology demonstration.

This all happened before



Samsung was previously caught faking benchmark scores for the Exynos 5 version of the Galaxy S4, as well as its Galaxy Note 3 using a Qualcomm Snapdragon chip and its Galaxy Tab 3 10.1 powered by an Intel CloverTrail+ processor.

This kind of benchmark fudging is a practice opposed by Qualcomm and Intel, as noted by AnandTech, which called attention to the benchmark doping in its testing this summer.

Popular benchmark vendor AnTuTu announced a new version of its app designed to work around cheating by Samsung and other companies early last month.

Samsung has issued multiple, contradictory denials that it is doing anything wrong, and certain Android fan sites have defended the practice.



While the difference in numbers that such cheating produces appears to be small, it allows Samsung to claim, for example, a performance edge over Apple's iPhone 5c with its own Galaxy S 4 flagship model, when in reality the S4 is about the same or scores slightly lower (above) when it is tested without being able to cheat.

When allowed to cheat, Samsung's Galaxy S4 appears to be nearly as fast as Apple's higher end iPhone 5s, and the Galaxy Note III appears to be faster, despite actually being slower in real world tasks, even though it has more memory, a larger battery and is set to run at a faster clock speed.
post #2 of 165
Just imagine the uproar if Apple did this. We'd never hear the end of it.
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #3 of 165
We can expect no less than cheating from manufacturers when their entire Android borg is based upon theft and deceit.
post #4 of 165

So when is the government going to get involved here?

Originally posted by Marvin

Even if [the 5.5” iPhone exists], it doesn’t deserve to.
Reply

Originally posted by Marvin

Even if [the 5.5” iPhone exists], it doesn’t deserve to.
Reply
post #5 of 165

That sort of behavior is beyond silly. You can expect if not accept some tiny start ups to do all sorts of crazy guerrilla marketing ideas to try and stand out and get some attention and traction but not from multi-billon dollar corporations. That is just sleazy. I hope all these benchmarking apps manage to thwart their cheating attempts by updating their apps to avoid the over clock cheat. 

 

At the end of the day unless all you do is run benchmarking apps they really don't mean much. I have to admit I am curious to see how my 5s stacks up to the competition but only if the tests are not skewed by over clocking. I would also be very curious to see actual day to day type of benchmark tests that measure common tasks and apps. 

 

Never argue with an idiot. They will only bring you down to their level and beat you with experience. 

Reply

 

Never argue with an idiot. They will only bring you down to their level and beat you with experience. 

Reply
post #6 of 165

I'm curious to see what insane scores a doped up iPhone 5S could crank out.

post #7 of 165
Zero integrity
post #8 of 165
It's not so much like an athlete doping (makes it sound like a "cheat" that users would be glad to have); it's more like an athlete rigging a fake measurement of their running speed... when in a real game, they can't run that fast and end up losing!
post #9 of 165

Samsung:  "We did not use any performance-enhancing techniques"  Sounds vaguely familiar in other performance arenas.  Maybe Samsung can get some of these similarly discredited athletes to testify for them.

post #10 of 165
Quote:
Originally Posted by Bagman View Post

Samsung:  "We did not use any performance-enhancing techniques"  Sounds vaguely familiar in other performance arenas.  Maybe Samsung can get some of these similarly discredited athletes to testify for them.

Or just start naming their devices after them, the Samsung ARod lol.gif
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
post #11 of 165
Stay classy Samsung. Too funny...
post #12 of 165
Quote:
Originally Posted by Tallest Skil View Post

So when is the government going to get involved here?

Less government not more.
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
post #13 of 165
Quote:
Originally Posted by Ireland View Post

Just imagine the uproar if Apple did this. We'd never hear the end of it.

 

That's just it... Apple has no need to do this. ;)

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #14 of 165
I think they should do it the other way around: down clock it and surprise the user with a way faster phone than they thought they were getting. Oh well, this way they'll surprise the customer also. And it's a sale. Chalk one up.
post #15 of 165
Quote:
Originally Posted by Tallest Skil View Post
 

So when is the government going to get involved here?

 

Phoneblock! Phoneblock! Phoneblock!

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #16 of 165
Quote:
Originally Posted by PhilBoogie View Post

I think they should do it the other way around: down clock it and surprise the user with a way faster phone than they thought they were getting. Oh well, this way they'll surprise the customer also. And it's a sale. Chalk one up.

That makes too much sense.
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
post #17 of 165
Quote:
Originally Posted by Tallest Skil View Post

So when is the government going to get involved here?

Which government? I assume you mean the US NIST Office of Weights and Measure?

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #18 of 165
Once you are a cheater you are always cheater. Samsung and the entire Android culture (copy and steal) starts with Google's CEO Eric Schmidt! Glad they got caught. Let their fans to defend their beloved products, in the end of they are the stupid ones got cheated.
post #19 of 165
Quote:
Originally Posted by Suddenly Newton View Post


Which government? I assume you mean the US NIST Office of Weights and Measure?

 

Or the US Dept. of Cheats & Liars?

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #20 of 165
Samsung copies designs, wilfully violates patents, and fakes benchmarks. Charming.
post #21 of 165
Quote:
Originally Posted by Suddenly Newton View Post

Which government? I assume you mean the US NIST Office of Weights and Measure?

This is nothing that can't be handled in Civil Court if Futuremark or, say Apple can prove harm. Futuremark possibly- violation of terms of license ; Apple - might be able to prove loss due to false advertising.

Reckon either thinks it's worth it?
post #22 of 165
Quote:
Originally Posted by Ireland View Post

Just imagine the uproar if Apple did this. We'd never hear the end of it.

 

Sure, Apple is a total stranger to benchmark cheating : http://www.theregister.co.uk/2003/06/24/apple_accused_of_cheating_over/ written by the same author.

post #23 of 165

I have yet to see this story appear on places like Engadget or TheVerge, no doubt they don't want their forums to explode due to Fandroids claiming it's acceptable

 

I suspect Scott Wilson will defend it (and blame Apple for doing it) should it ever appear on CNet

post #24 of 165
Quote:
Originally Posted by tooltalk View Post
 

 

Sure, Apple is a total stranger to benchmark cheating : http://www.theregister.co.uk/2003/06/24/apple_accused_of_cheating_over/ written by the same author.

I notice the article is over 10 years old (24th June 2003), obviously you had to dredge deep to find that one

post #25 of 165
I don't think it's an issue of again, just that Futuremark has finally decided to delist the cheaters from their site.

PS: Check out ddriver's comments at AnandTech on how this is all an evil plot by Apple. He must be chummy with tooltalk.

Edited by SolipsismX - 11/27/13 at 3:11pm

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply
post #26 of 165
Quote:
Originally Posted by PJWilkin View Post

I notice the article is over 10 years old (24th June 2003), obviously you had to dredge deep to find that one

And if you read the article it doesn't really show Apple blatantly cheating, suggests that other vendors are cheating too, and even states that at least Apple notes how it's choosing its benchmarks, but most of all it has nothing to do with anything to do with this article.

IOW, there is no clear demarcation with that decade old article that shows Apple's OS detecting specific benchmarks to run at artificially high speeds that are otherwise not allowed by typical apps under any circumstances. It's just the same as what Apple and every vendor does now when they chose the benchmark results that look best for marketing.
Edited by SolipsismX - 11/27/13 at 3:21pm

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply
post #27 of 165
Quote:
Originally Posted by PJWilkin View Post

I notice the article is over 10 years old (24th June 2003), obviously you had to dredge deep to find that one

So it was ok to do it 10 years ago?
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
post #28 of 165
how do you know apple is not cheating, clearly you cant see anything that goes on on the cpu
post #29 of 165
Quote:
Originally Posted by PJWilkin View Post
 

I have yet to see this story appear on places like Engadget or TheVerge, no doubt they don't want their forums to explode due to Fandroids claiming it's acceptable

 

I suspect Scott Wilson will defend it (and blame Apple for doing it) should it ever appear on CNet

 

/ahem

Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
post #30 of 165
Quote:
Originally Posted by Wilson Dur View Post
 

keep drinking you apple juice 


Stop calling me apple juice!

 

Oh...

post #31 of 165
Quote:
Originally Posted by Wilson Dur View Post

how do you know apple is not cheating, clearly you cant see anything that goes on on the cpu

Educate yourself…

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply
post #32 of 165
Quote:
Originally Posted by SolipsismX View Post

I don't think it's an issue of again, just that Futuremark has finally decided to delist the cheaters from their site.

PS: Check out ddriver's comments at AnandTech on how this is all an evil plot by Apple. He must be chummy with tooltalk.

So it's Apple that is paying benchmark companies to delist cheaters, huh? Do these fandroids want desperately to believe in their fantasy benchmarks so they can feel good about whatever phone they bought?

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply

"Apple should pull the plug on the iPhone."

John C. Dvorak, 2007
Reply
post #33 of 165
Quote:
Originally Posted by Suddenly Newton View Post


So it's Apple that is paying benchmark companies to delist cheaters, huh? Do these fandroids want desperately to believe in their fantasy benchmarks so they can feel good about whatever phone they bought?

 

It seems so, judging by some of the comments and one star reviews here.

 

These Android guys can't handle anything detracting from their decision to purchase also ran products.

 

How to shoot an Android guy down using an iPhone 5s.

 

 Hold your phone next to theirs, open camera apps together, put latest and greatest Samsung Note 3 or S4 in burst mode then hold down camera icon on both phones...

 

...BOOM headshot, they will be left in a whimpering mess as this demonstration of 64bit power blows them away.

Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
post #34 of 165
Quote:
Originally Posted by Suddenly Newton View Post

So it's Apple that is paying benchmark companies to delist cheaters, huh? Do these fandroids want desperately to believe in their fantasy benchmarks so they can feel good about whatever phone they bought?

It's one thing for someone to think systems throttling when they sense different benchmarks is the same as Turbo Mode in a CPU. It's completely ignorant but I can see how the les than astute person can be confused by the very real differences between them. But to blame Apple for other vendors artificially making their benchmarks look better is just crazy.

The worst part of all these vendors cheating is it only helps by a small percentage point, and not enough to make or break a sale, but probably enough to lose even more trust in that vendor. Moto, Asus and Apple are the prominent ones that don't seem to have cheated.

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply
post #35 of 165

Any reasonable person who reads your entire linked article would not conclude that Apple cheated 10 years ago. The last three paragraphs sum it up perfectly. 

 

Are you suggesting that NTC and Samsung are not cheating in these benchmarks? Or would you prefer to distract everyone with 10 year old stories? 

 

I'll make you a deal, I'll stay on topic if you will.... deal?

post #36 of 165
Originally Posted by dasanman69 View Post
Less government not more.

 

Completely and utterly irrelevant here. This is literally the government’s job: to protect its citizens. Enough of your purposefully confrontational BS.

Originally posted by Marvin

Even if [the 5.5” iPhone exists], it doesn’t deserve to.
Reply

Originally posted by Marvin

Even if [the 5.5” iPhone exists], it doesn’t deserve to.
Reply
post #37 of 165
Quote:
Originally Posted by Tallest Skil View Post

Completely and utterly irrelevant here. This is literally the government’s job: to protect its citizens. Enough of your purposefully confrontational BS.

I thought you were joking because if Apple was doing this there would be some Senator that would expect Tim Cook to personally tell him why Apple does this (and then he'd ask him why he can't see his grandkids shared photos in the Camera app).

This is not something the government will ever be involved with. The companies that manage the tests will do a fine job of killing results from anyone that dopes their scores but even then the vendor does has a right to alter how their system work on different conditions so this is purely an ethical and not a legal issue.

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply
post #38 of 165
Quote:
Originally Posted by SolipsismX View Post

The worst part of all these vendors cheating is it only helps by a small percentage point, and not enough to make or break a sale, but probably enough to lose even more trust in that vendor. Moto, Asus and Apple are the prominent ones that don't seem to have cheated.

Then why do it? What benefit does cheating a little known test have? The upside is minimal.
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
post #39 of 165
Quote:
Originally Posted by dasanman69 View Post

Then why do it? What benefit does cheating a little known test have? The upside is minimal.

They clearly think there is an upside that outweighs the downsides but I certainly don't see it. It might the just be immature programmers that think that also think that speeding for a few miles down an open highway is worth a ticket costing a few hundred dollars. I say this because it's not universally done across any vendor and there are clearly a lot more phones that don't employ this foolish technique, however it's almost always the higher-end and/or devices with the most mindshare coming out of the gate that have shown to be doped for the benchmark tests.

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply

"The real haunted empire?  It's the New York Times." ~SockRolid

"There is no rule that says the best phones must have the largest screen." ~RoundaboutNow

Reply
post #40 of 165
Quote:
Originally Posted by Tallest Skil View Post

Completely and utterly irrelevant here. This is literally the government’s job: to protect its citizens. Enough of your purposefully confrontational BS.

From cheated on benchmark tests? You expect a bunch of cheaters to catch other cheaters?
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
"Few things are harder to put up with than the annoyance of a good example" Mark Twain
"Just because something is deemed the law doesn't make it just" - SolipsismX
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Samsung Galaxy Note, HTC One caught cheating in benchmarks again