Apple's Benchmarks misleading?

1356789

Comments

  • Reply 41 of 178
    Quote:

    Originally posted by Flounder

    I'm actually looking forward to a mag like PC world putting it through its paces. They've actually been giving macs a pretty fair shake lately.



    Also, who gives a rat's ass about SPEC?

    I mean, apple can rig things to a certain extent in the application tests I'm sure. But when the G5 smokes the competition THAT much, in THAT many programs, there's something to it.




    By the time these G5's are ACTUALLY released in Sept 03 and we see an indepentant benchmark test in the unbiased Oct PC magazines the New Apple G5s WILL NOT BE THE FASTEST personal computers, nor will they give the best bang for the buck nor will they have the best speed to cost ratio. (This is normal for Apple)



    I am a HUGE Mac fan, but I don't appreciate deceptive marketing practices and fudging benchmark tests to claim to have the worlds fastest PC. The G5 is not the fastest PC right now and it for sure won't be when it is released this fall.



    BUT I still WANT one !



    -tom w
  • Reply 42 of 178
    from the article:



    Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices.



    omfg! thats rediculous! that practice has been done since the beginning of ****ing time. wow
  • Reply 43 of 178
    jobesjobes Posts: 106member
    agreed. real-world tests will count for most users. give me cinema 4d (or cinebench), lightwave, photoshop, after effects and cubase sx specs. all 3rd party, cross platform apps which many of us depend on for our livelihoods.



    for me cinema 4d and photoshop results alone will swing any purchasing decisions i have in the next six months, for work or home. I believe the g5 has largely levelled the playing field, and having Altivec (a bloody good SIMD unit) onboard could give a massive performance boost, either from existing G4 or intel processors. i just hope that, for the sake of developers and the platform in general, Apple really have made it easy to use altivec, and to recompile for 64 bits. i appreciate that many developers have been slow to grasp the benefits of altivec, especially if they are recompiling code from x86 and the necessary hooks aren't there for vector processing. apple's evangelism of altivec, and soon for the importance of 64bit native apps for certain sectors will be crucial to really see the potential of the g5 architecture.



    i'm glad to see Wolfram in the fore of development ... that Theo dude was a class-act geek onstage and they are passionate about running their code fast and efficiently. it's just lucky they have a long relationship with apple. i expect other developers with long histories of developing apps on multiple platforms, such as maxon, to reap early results by optimising their existing codebase for the new boxes.



    Just remember, SPEC results only ever tell part of the story.... no matter what way you read 'em
  • Reply 44 of 178
    klinuxklinux Posts: 453member
    detractor? So anyone who disagrees with Apple marketing is a troll or idiot? There is a reason why many people think Mac users are ignorant fanatics so some people in this thread illustrates this.



    For years, Steve tells us that megahertz is not important yet he shows G4 trouncing P4 in certain benchmarks. Most professionals dismissed that but some people still swears by them.



    Now G5 is on the scene, Steve is now saying megahertz is important and he shows G5 trouncing Xeon. So BOTH G4 and G5 are faster than their Intel counterparts? I do not believe that for a second.



    In addition, he is comparing a system that will not ship in three months versus one that is shipped three (or more) months ago. If anyone in the PC did that we would cry wolf but when Steve does it, it's OK?



    And why no comparison to AMD chips? The Itanium? The Opteron? How about the 800Mhz FSB P4 with HT (not those that were announced yesterday but that would be fair too) that are faster than the single P4 used for the demo ?



    I see that more and more dispute to Apple's benchmark will come in the next few days and definitely after the system ships.



    Hey Steve: Just let us know that OS X will be running on a much much better hardware OK!?
  • Reply 45 of 178
    johnhenryjohnhenry Posts: 152member
    Why wasn't the latest G4 included in the test too? I, for one, would have liked to see what performance boost I would get from a current machine on the apple side. I guess they were gearing it for the small percentage of PC people watching the keynote? Or just fueling the fire?
  • Reply 46 of 178
    Quote:

    Originally posted by klinux

    detractor? So anyone who disagrees with Apple marketing is a troll or idiot? There is a reason why many people think Mac users are ignorant fanatics so some people in this thread illustrates this.



    For years, Steve tells us that megahertz is not important yet he shows G4 trouncing P4 in certain benchmarks. Most professionals dismissed that but some people still swears by them.



    Now G5 is on the scene, Steve is now saying megahertz is important and he shows G5 trouncing Xeon. So BOTH G4 and G5 are faster than their Intel counterparts? I do not believe that for a second.



    In addition, he is comparing a system that will not ship in three months versus one that is shipped three (or more) months ago. If anyone in the PC did that we would cry wolf but when Steve does it, it's OK?



    And why no comparison to AMD chips? The Itanium? The Opteron? How about the 800Mhz FSB P4 with HT (not those that were announced yesterday but that would be fair too) that are faster than the single P4 used for the demo ?



    I see that more and more dispute to Apple's benchmark will come in the next few days and definitely after the system ships.



    Hey Steve: Just let us know that OS X will be running on a much much better hardware OK!?




    YES



    I am a Mac fan, but I am not fanatic.



    I loath the Windoze OS and I am not a fan of the M$ /Gates marketing practices, but I am ever more disgusted that Apple/Jobs would attempt to deceive the party faithful with cheap parlor tricks to show he is now sporting the fastest PC.



    I agree with the above post, klinux makes a great point there are PC's shipping right now that are faster than the soon to be DP G5. But Jobs did not test his latest creation agains the "AMD chips? The Itanium? The Opteron? How about the 800Mhz FSB P4 with HT "



    Well I guess the speculation is just begining as to how the G5's will perform when tested by unbiased labs and testers in the fall.



    -tom w
  • Reply 47 of 178
    scottscott Posts: 7,431member
    I'm just glad Apple got to the point where we can argue about this stuff. Better to have a mixed victory than a clear lose.
  • Reply 48 of 178
    All this is complete crap! Face it,was there any test apple could have run on that stage that pc nuts or paranoid apple fans would have accepted as truth? And you are griping about spec for crying out loud! That @#^% test is infamous for underreporting the real performance of the ppc and exagerating the performance of x86-especially intel.Hell,spec underreports the real world performance of the athalon and p3 relative to the p4.If spec was the final word on performance the G4 would be only as fast as a similar clocked p4! And this is absolute rubbish-no i am not talking abt altivec either.The test is too compiler dependent and ripe for cheating-and I suspect the test was never actually fair to begin with considering intel's undue influence in the industry.The only reason they showed spec scores was because the 970 is so powerful it can overcome some of the bias inherent in the suit.
  • Reply 49 of 178
    rickagrickag Posts: 1,626member
    I don't know if these have been posted, but they provide a lot of pertinant information.



    G5 White Paper - test methods mostly



    G5 White Paper - techs on the G5 Computer



    G5 White Paper - techs on the IBM 970



    Maybe some people could construe Apple's results on spec as being skewed, because of the compiler used, but this is how the game is played.



    But the application results are pretty damn apparent. Apple used 45 effects in their 100 step Photoshop bakeoff. Maybe slightly in their favor, but people THE DUAL 2.0GHz. MACHINE WAS TWICE AS FAST. Let me repeat TWICE AS FAST. There is NO AMOUNT OF SKEWING that can lead me to believe that the new Apple computer isn't the fastest.



    The Mathmatica backoff is even more telling. AGAIN over twice as fast.



    Just for grins they threw in Blast, highly optimized for Altivec and well just read the results.



    Look up the new Quicktime movie that has the Apple team, with Adobe and Mathmatica reps on it. This is the real deal, and it has only just begun.



    Holy crap 3.0GHz next year, believe it. We aren't dealing with Motorola anymore. IBM believes in this tech, is guarding their server turf from interlopers and they intend to win.



    For years Windows fans have flaunted spec scores over Apple, well so what. All this proves is that the compilers used in spec benchs can have a more significant effect on the scores than the cpu's architecture - phewy forget spec. I don't run spec on my computer, it doesn't do anything, nothing nadda. NO ONE MAKES A LIVING RUNNING SPEC EXECPT SPEC.



    me take deep breath, calms down, rant over. you may now return to bashing Apple for playing Intels game at spec bs.



    have a nice day



  • Reply 50 of 178
    jlljll Posts: 2,713member
    Quote:

    Originally posted by klinux

    For years, Steve tells us that megahertz is not important yet he shows G4 trouncing P4 in certain benchmarks. Most professionals dismissed that but some people still swears by them.



    Now G5 is on the scene, Steve is now saying megahertz is important and he shows G5 trouncing Xeon.




    MHz important? The G5 still shows that MHz isn't everything.
  • Reply 51 of 178
    wolfeye155wolfeye155 Posts: 425member
    I agree considering the G5 processor is 1 GHz less than the 3 GHz Xeon!
  • Reply 52 of 178
    bodhibodhi Posts: 1,424member
    Current Hardware....
  • Reply 53 of 178
    bill mbill m Posts: 324member
    I think this is all good. The potential switchers in the PC camp finally have a formidable Apple hardware option and Panther should make it even tougher to resist. PC users that take hardware components into consideration, will certainly realize that the new G5 pricing scheme is fair and attractive to say the least. Apple is a top of the line manufacturer and well known for its quality offerings. Mediocre performance is certainly a thing of the past now.



    Some PC zealots are running like crazy trying to dismiss and shoot down 970/G5 speed claims, mainly due to the fact that their (PC) speed king throne probably has or will be abolished soon. Independent real world tests should be around in weeks, and I have no doubts we are in for a pleasant surprise.



    On the other hand, Spec tests mean nothing to consumers, compared to MHz and princing. I don't think these tests should have taken the center stage during Jobs' keynote, but after all, it was WWDC. I also think Apple's new marketing efforts should steer away from the Specs thingie, because it only seems to confuse the facts. The MHz rating (myth) is what really counts to consumers and it seems that IBM will be able to shorten that gap sooner than expected. Apple has realized this and will no doubt address this in the near future given its new processor options. I expect consumer towers sporting a single G5 with top speed ratings and some components shortcomings for similar prices to their PC counterparts at any given MHz rating.



    The G4 and Motorola's manufacturing issues were hurting Apple badly, image and performance wise. That chapter seems to be over. For good. Welcome to the new performance leader. Speed and computing experience beyond anything else on the market.
  • Reply 54 of 178
    klinuxklinux Posts: 453member
    Quote:

    Originally posted by JLL

    MHz important? The G5 still shows that MHz isn't everything.



    You are correct. I should have said "speed" in the general term.



    But yes, what would have truly impressed me is a G5 to G4 benchmark.



    Also, for real-world benchmarking, I like how Tom's Hardware does it in that it tests five areas and I think that is fair to people who use the machines for game vs AV vs work etc.



    # OpenGL Benchmarks

    # Direct3D Benchmarks

    # Audio/Video Benchmarks

    # Applications Benchmarks

    # Synthetic Benchmarks





    Lastly, by the time the G5 ships, Intel will likely have their 90nm Prescott CPU ready (the so-called Pentium 5) and we will have 16 threads every other week covering PC vs Mac speed. Oy!



    All I really want is the G5 (the middle one).
  • Reply 55 of 178
    Quote:

    Originally posted by cuneglasus

    All this is complete crap! Face it,was there any test apple could have run on that stage that pc nuts or paranoid apple fans would have accepted as truth?



    The crux of the argument here isn't whether or not SPEC is a good test or not, but that Apple is intentionally misleading in their claims.



    I find nothing wrong with the Haxial website, and what he's done. He hasn't taken Apple at face value, and checked the facts. What's wrong with that? I don't believe he's 100% correct, but checking facts is A Good Thing™.



    Others have pointed out that companies play this game a lot. nVidia was just in the news regarding their drivers, for example. In the Register's article, they were at least complementing Apple (sort of) for posting their test methods, while they're not entirely sure of Dell's methods.



    So while I think it's valid to question Apple's "Fastest Computer" claim, at the end of the day, do you want one of these whether or not it's truly the fastest?



    Of course you do. Unless you're Clive.
  • Reply 56 of 178
    dobbydobby Posts: 797member
    I want to see the benchmarks with 64bit optimized code.

    A 64bit Cpu running a 32bit app isn't running at its maximum. When it does we can compare the G5 (or 970 etc) with Itanic.



    The benchmarks should also include more relevant things like boot times, app opening times etc. Thats something I can relate to.



    Dobby.
  • Reply 57 of 178
    yevgenyyevgeny Posts: 1,148member
    The GCC comparison is valid in that it would seem to show that a G5 is faster running code compiled with GCC 3.3 than a P4 or Xeon is running code compiled with GCC.



    Some people say that this is misleading. It can be somewhat misleading, unless you are a linux developer and you use GCC to compile your software, in which case it is very relevant to you.



    Even if people don't believe the SPEC benchmarks (and like all benchmarks, they should be taken with caution because nobody's benchmarks are clean), then do believe the head of development for Wolfram when he says that the G5 rocks. He has no motivation to lie.



    Folks, the G5 is a kick ass machine. I am sure that there are some scenarios where the P4 beats it. I am sure that there are many scenarios where the G5 beats Intel.
  • Reply 58 of 178
    rageousrageous Posts: 2,170member
    Quote:

    Originally posted by Yevgeny

    Folks, the G5 is a kick ass machine.



    © "the G5 is a kick ass machine" is copyright 2003 Apple Computer Inc.
  • Reply 59 of 178
    yevgenyyevgeny Posts: 1,148member
    Quote:

    Originally posted by GardenOfEarthlyDelights

    The crux of the argument here isn't whether or not SPEC is a good test or not, but that Apple is intentionally misleading in their claims.



    Everyone lies with their claims. Everybody. No exceptions. Or, perhaps better stated, everyone stretches the truth of their claims. This is standard industry practice. It is called marketing. Apple was kind enough to do it in a transparent way.



    Last week, I posted the progression of lies in the software business (common knowledge at the company that I work at):



    Lies

    Damn Lies

    Statistics

    Benchmarks

    Hardware release dates

    Software release dates



    Really folks, benchmarks are only more truthful than hardware and software release dates.
  • Reply 60 of 178
    eugeneeugene Posts: 8,254member
    Quote:

    Originally posted by Powerdoc

    The G5 is on the more modern mobo of the time, with 128 wide (and not dual channel) 400 mhz DDR ram, with hypertransport link, Sata disk, PCI-X slots.



    IBM and Apple must suck terribely if with such features , they canno't compare with the X-86 world.




    The G5 mobo DOES use dual-channel DDR memory, and that's a good thing.
Sign In or Register to comment.