Samsung Galaxy Note, HTC One caught cheating in benchmarks again

Posted:
in iPhone edited January 2014
Futuremark has delisted the scores of Samsung and HTC products from its public benchmark listings over new allegations of performance doping.



According to a report by Tony Smith of the Register, Futuremark accused the two companies of rigging results in its 3DMark GPU benchmarking app for Android.

The company removed the scores of Samsung's Galaxy Note 10.1 tablet and Note III phablet as well as HTC's One and One Mini smartphones, dropping the models to the bottom of its listings, below devices with a score of zero (above).

Futuremark President Oliver Baltuch told the site, "People rely on Futuremark benchmarks to produce accurate and unbiased results. That's why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software.

Baltuch explained, "in simple terms, a device must run our benchmarks without modification as if they were any other application."

Futuremark's rules stated "the platform may not detect the launch of the benchmark executable" and that devices "must not alter, replace or override any parameters or parts of the test, nor modify the usual functioning of the platform based on the detection of the benchmark."

The leading device in Futuremark's listings remains the Samsung Galaxy Round, version of the Note III with a curved body and screen and lacking a stylus. The low volume product is offered in Korea for close to $1000, essentially as a technology demonstration.

This all happened before


Samsung was previously caught faking benchmark scores for the Exynos 5 version of the Galaxy S4, as well as its Galaxy Note 3 using a Qualcomm Snapdragon chip and its Galaxy Tab 3 10.1 powered by an Intel CloverTrail+ processor.

This kind of benchmark fudging is a practice opposed by Qualcomm and Intel, as noted by AnandTech, which called attention to the benchmark doping in its testing this summer.

Popular benchmark vendor AnTuTu announced a new version of its app designed to work around cheating by Samsung and other companies early last month.

Samsung has issued multiple, contradictory denials that it is doing anything wrong, and certain Android fan sites have defended the practice.



While the difference in numbers that such cheating produces appears to be small, it allows Samsung to claim, for example, a performance edge over Apple's iPhone 5c with its own Galaxy S 4 flagship model, when in reality the S4 is about the same or scores slightly lower (above) when it is tested without being able to cheat.

When allowed to cheat, Samsung's Galaxy S4 appears to be nearly as fast as Apple's higher end iPhone 5s, and the Galaxy Note III appears to be faster, despite actually being slower in real world tasks, even though it has more memory, a larger battery and is set to run at a faster clock speed.
«13456789

Comments

  • Reply 1 of 169
    irelandireland Posts: 17,547member
    Just imagine the uproar if Apple did this. We'd never hear the end of it.
  • Reply 2 of 169
    We can expect no less than cheating from manufacturers when their entire Android borg is based upon theft and deceit.
  • Reply 3 of 169

    So when is the government going to get involved here?

  • Reply 4 of 169
    gwmacgwmac Posts: 1,796member

    That sort of behavior is beyond silly. You can expect if not accept some tiny start ups to do all sorts of crazy guerrilla marketing ideas to try and stand out and get some attention and traction but not from multi-billon dollar corporations. That is just sleazy. I hope all these benchmarking apps manage to thwart their cheating attempts by updating their apps to avoid the over clock cheat. 

     

    At the end of the day unless all you do is run benchmarking apps they really don't mean much. I have to admit I am curious to see how my 5s stacks up to the competition but only if the tests are not skewed by over clocking. I would also be very curious to see actual day to day type of benchmark tests that measure common tasks and apps. 

  • Reply 5 of 169

    I'm curious to see what insane scores a doped up iPhone 5S could crank out.

  • Reply 6 of 169
    Zero integrity
  • Reply 7 of 169
    nagrommenagromme Posts: 2,834member
    It's not so much like an athlete doping (makes it sound like a "cheat" that users would be glad to have); it's more like an athlete rigging a fake measurement of their running speed... when in a real game, they can't run that fast and end up losing!
  • Reply 8 of 169
    bagmanbagman Posts: 349member

    Samsung:  "We did not use any performance-enhancing techniques"  Sounds vaguely familiar in other performance arenas.  Maybe Samsung can get some of these similarly discredited athletes to testify for them.

  • Reply 9 of 169
    dasanman69dasanman69 Posts: 12,976member
    bagman wrote: »
    Samsung:  "We did not use any performance-enhancing techniques"  Sounds vaguely familiar in other performance arenas.  Maybe Samsung can get some of these similarly discredited athletes to testify for them.

    Or just start naming their devices after them, the Samsung ARod :lol:
  • Reply 10 of 169
    Stay classy Samsung. Too funny...
  • Reply 11 of 169
    dasanman69dasanman69 Posts: 12,976member
    So when is the government going to get involved here?

    Less government not more.
  • Reply 12 of 169
    Quote:

    Originally Posted by Ireland View Post



    Just imagine the uproar if Apple did this. We'd never hear the end of it.

     

    That's just it... Apple has no need to do this. ;)

  • Reply 13 of 169
    I think they should do it the other way around: down clock it and surprise the user with a way faster phone than they thought they were getting. Oh well, this way they'll surprise the customer also. And it's a sale. Chalk one up.
  • Reply 14 of 169
    Quote:

    Originally Posted by Tallest Skil View Post

     

    So when is the government going to get involved here?


     

    Phoneblock! Phoneblock! Phoneblock!

  • Reply 15 of 169
    dasanman69dasanman69 Posts: 12,976member
    philboogie wrote: »
    I think they should do it the other way around: down clock it and surprise the user with a way faster phone than they thought they were getting. Oh well, this way they'll surprise the customer also. And it's a sale. Chalk one up.

    That makes too much sense.
  • Reply 16 of 169
    So when is the government going to get involved here?

    Which government? I assume you mean the US NIST Office of Weights and Measure?
  • Reply 17 of 169
    Once you are a cheater you are always cheater. Samsung and the entire Android culture (copy and steal) starts with Google's CEO Eric Schmidt! Glad they got caught. Let their fans to defend their beloved products, in the end of they are the stupid ones got cheated.
  • Reply 18 of 169
    Quote:

    Originally Posted by Suddenly Newton View Post





    Which government? I assume you mean the US NIST Office of Weights and Measure?

     

    Or the US Dept. of Cheats & Liars?

  • Reply 19 of 169
    Samsung copies designs, wilfully violates patents, and fakes benchmarks. Charming.
  • Reply 20 of 169
    Which government? I assume you mean the US NIST Office of Weights and Measure?

    This is nothing that can't be handled in Civil Court if Futuremark or, say Apple can prove harm. Futuremark possibly- violation of terms of license ; Apple - might be able to prove loss due to false advertising.

    Reckon either thinks it's worth it?
Sign In or Register to comment.