"Greg Joswiak, vice president of hardware product marketing at Apple, in a phone interview today, defended Apple's performance claims for its upcoming Power Mac G5, after they came under fire in the wake of yesterday's announcement. Read on for the details.
Joswiak went over the points in turn, but first said that they set out from the beginning to do a fair and even comparison, which is why they used an independent lab and provided full disclosure of the methods used in the tests, which would be "a silly way to do things" if Apple were intending to be deceptive.
He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.
He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.
Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.
In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).
As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags). "
This is all academic anyway, since nobody can buy a G5 and see for themselves, and as the x86 fanatics are all too happy to point out, the x86 landscape will have changed slightly by the time the PowerMac G5 actually ships.
I cannot agree more with this statement (except maybe the "slightly" part but that is still irrelevant at this point)!
i don't think the benchmarks are really the important stats.
the most important is how much faster they are than what we own now.
i think it's obvious that most of us won't be buying pc's, so to compare them is like subaru comparing a WRX to a bmw M3.
not too relevant! the numbers never tell the whole story!
i don't care about the stats as long as it screams on OSX!
you can't have that on any pc, no matter the speed!
You could have a version of Linux with KDE/Gnome for the PC, which has many, if not all, of the interface features of OSX, and is also free.
Let's not forget that OSX is actually a free UNIX OS "under the hood" with an Apple window manager, not an original Apple creation.
Now that the Mac has a proper grown-up's OS, I would, as a Win2k PC user, consider buying a Mac. That is, if they weren't so expensive, especially in the UK.
After reading the mac-hater article from Haxial, I went about the internet to find some spec results, and below are my findings. There's a different list over at MacinTouch. You can take these numbers however you like, but I highly doubt that the numbers itself can be compared to each other on an objective basis (except for the two taken from Specs site).
PowerPC 970 1.8Ghz estimated (from IBM)
specint 937, specfp 1051
Intel P4 3.2Ghz (from Intel)
specint 1221, specfp 1252
AMD Athlon 3200+ (from Spec)
specint 1080, specfp 982
Dell PowerEdge 2650, 3.06 Xeon (from Spec)
specint 1056, specfp 1003
Apple's specs on the 2.0 Ghz PPC 970 (from Apple)
specint 800, specfp 840
Also Intel claims 406fps with Quake III v1.30 demo 4 (graphics card used unknown).
AMD claims 328fps with Quake III (version unknown) demo 2 (graphics card unknown-but listed)
Apple claims 337fps with Quake III v1.32 with ATI Radeon 9800 Pro 128MB card (demo used unknown)
Also Intel claims 406fps with Quake III v1.30 demo 4 (graphics card used unknown).
AMD claims 328fps with Quake III (version unknown) demo 2 (graphics card unknown-but listed)
Apple claims 337fps with Quake III v1.32 with ATI Radeon 9800 Pro 128MB card (demo used unknown)
Mac-hater? I thought the Haxial article was balanced and informative, and the quoted flames were riotously funny. My colleagues and I (working in blissful harmony in a Mac/PC design office) had a good laugh this morning at the tone and grammar of some of those emails.
Forget frames-per-second for a moment - I am interested in how you manage to play Quake without a 3 button mouse.
The author of the haxial.com report is hippocritical: he blames apple for pushing their own machine's tests, but then compares the G5 with banechmarks of AMD and Intel processors that were done by AMD and Intel, (in that order).
I can't believe how many people are complaining about the G5's. I mean who cares Apple is using doctored benchmarks, if this thing is able to make OSX purr then I'm game. Yes they are expensive and are still not as fast as Intel's offering but I don't know about you I use Mac because it best works for me In terms of design, operating system, support, software and the smaller more elite community. I'm the guy who buys Bang & Olufsen stereos not because their the most advanced, I buy them because of their awesome design and simple controls. Turn it on and it works, ALWAYS! Apple to me is the B & O of computers, beautiful, elegant and simple.
Mac-hater? I thought the Haxial article was balanced and informative, and the quoted flames were riotously funny.
I guess you haven't read his "All-Mac-users-that-don't-like-the-Haxial-interface-are-Mac-fanatics-and-Apple-copied-Windows-anyway" article.
My colleagues and I (working in blissful harmony in a Mac/PC design office) had a good laugh this morning at the tone and grammar of some of those emails.
Quote:
Originally posted by computer_user
I am interested in how you manage to play Quake without a 3 button mouse.
Perhaps they buy the mouse they want? My MX700 works fine on my Mac.
The funny thing was how the Haxial dude claimed from the get go to be a Mac user. Now THAT was funny. Almost as funny was how he seemed to be gaining "momentum" the longer the article went. Halfway through the article I could sense he felt he was on a roll and forgot he was supposed to be a "disappointed and betrayed Mac user" . If that guy's a Mac user, I'll do a JD.
P.s That idiot was actually whining about Apple's pricing too...
"Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices."
LMAO, I hope I never run into this idiot at the market!
If you look into it, they ripped nada off. They only used the idea and made it into an workable OS, Xerox was basically going to "can it".
As Homer Simpson would say: doh!
Yes, as I understand it, the "suits" at Zerox (sorry, Xerox!), couldn't quite grasp what their tech people had come up with and told them to drop it. Bet they kicked themselves when they saw what Apple and Microsoft did with it...
The crux of the argument here isn't whether or not SPEC is a good test or not, but that Apple is intentionally misleading in their claims.
I find nothing wrong with the Haxial website, and what he's done. He hasn't taken Apple at face value, and checked the facts. What's wrong with that? I don't believe he's 100% correct, but checking facts is A Good Thing™.
Others have pointed out that companies play this game a lot. nVidia was just in the news regarding their drivers, for example. In the Register's article, they were at least complementing Apple (sort of) for posting their test methods, while they're not entirely sure of Dell's methods.
So while I think it's valid to question Apple's "Fastest Computer" claim, at the end of the day, do you want one of these whether or not it's truly the fastest?
Of course you do. Unless you're Clive.
I beg to differ. For a decent review of the tests go to The Register, they seem to have a very even handed article that explains why the tests were under the way they were. Apple was not intentional misleading any one and neither was the testing service.
Boy, a crowd of insecure PC users sure has turned this forum to shit rather quickly.
The only thing we can wait for now is for IBM to release SPEC numbers based off of their own compiler. Once they've done that, they can be compared to Intel's own 'fixed' numbers.
Actually Apple bought the GUI "Idea" from Xerox...
Xerox didn't think they had anything and Apple knew they did and jumped all over it.
It was Microsoft that saw Apple's implementation; which was much different that Xerox's prototype; and ripped Apple off.
Xerox planted the seed. Apple nurtured it into a fruit bearing tree. And it was Microsoft that snuck into the orchard and stole the fruit from the tree.
Regarding the Haxial article being "balanced and informative"...
1 - How ironic to call the article "balanced"...when the author bases the majority of his arguments against Apple on the fact that Apple attempted to BALANCE the tests via the use of GCC rather than to allow the test to be skewed by specifically optimized compilers on EITHER side.
2 - Was it informative...well Apple's Joswiak (as well as information posted on Dell's own web site regarding their own SPEC tests with HyperTransport) seems to blow at least 80% of the Haxial article's "facts" out of the water. The guy wrote something less than 24 hours after the G5 keynote that was based on an inaccurate understanding and view of the test results. Therefore, all of the "conclusions" he made regarding the test were skewed. Yet HIS comments are taken as fact by the PC world, while Apple is the big "liar'.
I'd like a PC user to answer this for me...
Apple's Single 2.0 GHz G5 SPEC Scores:
SPECint 800
SPECfp 840
IBM's Single 1.8 GHz G5 SPEC Scores:
SPECint 937
SPECfp 1051
According to this, Apple's own testing shows a 2.0 GHz G5 is supposedly 20% slower than the 1.8 GHz G5? Impossible? So how can you account for the difference in speeds? Say it with me...
BECAUSE APPLE RAN THEIR BENCHMARKS BY TRYING TO USE THE MOST BALANCED PLATFORM SETTINGS ON BOTH SIDES, INCLUDING THE USE OF THE GCC COMPILER, IN ORDER TO LEVEL THE PLAYING FIELD AND PROVIDE THE MOST UNBIASED SPEC SCORES THAT ARE POSSIBLE!
If Apple had used whatever optimized compiler settings that IBM ran when they tested the 1.8 GHz G5, the 2.0 GHz would have been about 10% faster than IBM's scores. That would have made them around the 1037/1151 mark, give or take. Why would Apple have published a report with scores of 800/840 instead of 1037/1151? Their SPECint scores would have been 20% higher and their SPECfp scores would have been 30% higher!
The only explanation for the discrepancy is that Apple's tests were intentionally NOT optimized...intentionally skewed down by doing exactly what they claimed: Using GCC on both sides to provide as FAIR a playing field, without tricks, as possible. And it's the fact that the PCers refuse to concede this point that is most annoying.
Apple has NOT claimed that the PC SPEC scores can't be tricked into being higher through Intel compiler optimization...that wasn't the point of THIS particular test. Do your own friggin' tests! But under THIS test, the results are valid. And Apple's claim has not yet been refuted scientifically...only attacked by the PC masses without grasping the facts. And no matter how much some may whine, there is no evidence Apple lied or intentionally misled anyone.
Apple's scores being lower than IBM's seem proof to me that Apple didn't go out of their way to falsify the test results...if they did, it hurt their own scores between 20-30%. You have to be a major conspiracy theorist to believe THAT! I do think that Apple should do new tests where they show Windows results, Intel's compiler results, etc. And then let IBM do the Apple tests with an IBM compiler. Wait til we see the crying then...
Comments
Joswiak went over the points in turn, but first said that they set out from the beginning to do a fair and even comparison, which is why they used an independent lab and provided full disclosure of the methods used in the tests, which would be "a silly way to do things" if Apple were intending to be deceptive.
He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.
He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.
Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.
In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).
As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags). "
Originally posted by Amorph
This is all academic anyway, since nobody can buy a G5 and see for themselves, and as the x86 fanatics are all too happy to point out, the x86 landscape will have changed slightly by the time the PowerMac G5 actually ships.
I cannot agree more with this statement (except maybe the "slightly" part but that is still irrelevant at this point)!
Originally posted by klinux
I cannot agree more with this statement (except maybe the "slightly" part but that is still irrelevant at this point)!
Why not slightly? AFAIK Intel won't have anything new out by August.
Originally posted by whoami
i don't think the benchmarks are really the important stats.
the most important is how much faster they are than what we own now.
i think it's obvious that most of us won't be buying pc's, so to compare them is like subaru comparing a WRX to a bmw M3.
not too relevant! the numbers never tell the whole story!
i don't care about the stats as long as it screams on OSX!
you can't have that on any pc, no matter the speed!
You could have a version of Linux with KDE/Gnome for the PC, which has many, if not all, of the interface features of OSX, and is also free.
Let's not forget that OSX is actually a free UNIX OS "under the hood" with an Apple window manager, not an original Apple creation.
Now that the Mac has a proper grown-up's OS, I would, as a Win2k PC user, consider buying a Mac. That is, if they weren't so expensive, especially in the UK.
PowerPC 970 1.8Ghz estimated (from IBM)
specint 937, specfp 1051
Intel P4 3.2Ghz (from Intel)
specint 1221, specfp 1252
AMD Athlon 3200+ (from Spec)
specint 1080, specfp 982
Dell PowerEdge 2650, 3.06 Xeon (from Spec)
specint 1056, specfp 1003
Apple's specs on the 2.0 Ghz PPC 970 (from Apple)
specint 800, specfp 840
Also Intel claims 406fps with Quake III v1.30 demo 4 (graphics card used unknown).
AMD claims 328fps with Quake III (version unknown) demo 2 (graphics card unknown-but listed)
Apple claims 337fps with Quake III v1.32 with ATI Radeon 9800 Pro 128MB card (demo used unknown)
Originally posted by supernature
After reading the mac-hater...
Also Intel claims 406fps with Quake III v1.30 demo 4 (graphics card used unknown).
AMD claims 328fps with Quake III (version unknown) demo 2 (graphics card unknown-but listed)
Apple claims 337fps with Quake III v1.32 with ATI Radeon 9800 Pro 128MB card (demo used unknown)
Mac-hater? I thought the Haxial article was balanced and informative, and the quoted flames were riotously funny. My colleagues and I (working in blissful harmony in a Mac/PC design office) had a good laugh this morning at the tone and grammar of some of those emails.
Forget frames-per-second for a moment - I am interested in how you manage to play Quake without a 3 button mouse.
Originally posted by Placebo
The author of the haxial.com report is hippocritical: he blames apple for pushing their own machine's tests, but then compares the G5 with banechmarks of AMD and Intel processors that were done by AMD and Intel, (in that order).
I think you mean "hyppocritical".
Originally posted by supernature
Apple claims 337fps with Quake III v1.32 with ATI Radeon 9800 Pro 128MB card (demo used unknown)
Apple uses demo 4.
Originally posted by computer_user
Mac-hater? I thought the Haxial article was balanced and informative, and the quoted flames were riotously funny.
I guess you haven't read his "All-Mac-users-that-don't-like-the-Haxial-interface-are-Mac-fanatics-and-Apple-copied-Windows-anyway" article.
My colleagues and I (working in blissful harmony in a Mac/PC design office) had a good laugh this morning at the tone and grammar of some of those emails.
Originally posted by computer_user
I am interested in how you manage to play Quake without a 3 button mouse.
Perhaps they buy the mouse they want? My MX700 works fine on my Mac.
P.s That idiot was actually whining about Apple's pricing too...
"Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices."
LMAO, I hope I never run into this idiot at the market!
Originally posted by computer_user
I think you mean "hyppocritical".
And I think you mean "hypocritical"
Originally posted by JLL
I guess you haven't read his "All-Mac-users-that-don't-like-the-Haxial-interface-are-Mac-fanatics-and-Apple-copied-Windows-anyway" article.
As I understand it (perhaps I need an acronym here), both Microsoft and Apple ripped off the GUI concept from Zerox.
If you look into it, they ripped nada off. They only used the idea and made it into an workable OS, Xerox was basically going to "can it".
Originally posted by ZoranS
errr.. thats Xerox friend.
If you look into it, they ripped nada off. They only used the idea and made it into an workable OS, Xerox was basically going to "can it".
As Homer Simpson would say: doh!
Yes, as I understand it, the "suits" at Zerox (sorry, Xerox!), couldn't quite grasp what their tech people had come up with and told them to drop it. Bet they kicked themselves when they saw what Apple and Microsoft did with it...
Originally posted by GardenOfEarthlyDelights
The crux of the argument here isn't whether or not SPEC is a good test or not, but that Apple is intentionally misleading in their claims.
I find nothing wrong with the Haxial website, and what he's done. He hasn't taken Apple at face value, and checked the facts. What's wrong with that? I don't believe he's 100% correct, but checking facts is A Good Thing™.
Others have pointed out that companies play this game a lot. nVidia was just in the news regarding their drivers, for example. In the Register's article, they were at least complementing Apple (sort of) for posting their test methods, while they're not entirely sure of Dell's methods.
So while I think it's valid to question Apple's "Fastest Computer" claim, at the end of the day, do you want one of these whether or not it's truly the fastest?
Of course you do. Unless you're Clive.
I beg to differ. For a decent review of the tests go to The Register, they seem to have a very even handed article that explains why the tests were under the way they were. Apple was not intentional misleading any one and neither was the testing service.
The only thing we can wait for now is for IBM to release SPEC numbers based off of their own compiler. Once they've done that, they can be compared to Intel's own 'fixed' numbers.
Originally posted by dstranathan
Apple comments:
http://apple.slashdot.org/apple/03/0...id=126&tid=181
Thanks
that was an interesting post
-tom w
Xerox didn't think they had anything and Apple knew they did and jumped all over it.
It was Microsoft that saw Apple's implementation; which was much different that Xerox's prototype; and ripped Apple off.
Xerox planted the seed. Apple nurtured it into a fruit bearing tree. And it was Microsoft that snuck into the orchard and stole the fruit from the tree.
1 - How ironic to call the article "balanced"...when the author bases the majority of his arguments against Apple on the fact that Apple attempted to BALANCE the tests via the use of GCC rather than to allow the test to be skewed by specifically optimized compilers on EITHER side.
2 - Was it informative...well Apple's Joswiak (as well as information posted on Dell's own web site regarding their own SPEC tests with HyperTransport) seems to blow at least 80% of the Haxial article's "facts" out of the water. The guy wrote something less than 24 hours after the G5 keynote that was based on an inaccurate understanding and view of the test results. Therefore, all of the "conclusions" he made regarding the test were skewed. Yet HIS comments are taken as fact by the PC world, while Apple is the big "liar'.
I'd like a PC user to answer this for me...
Apple's Single 2.0 GHz G5 SPEC Scores:
SPECint 800
SPECfp 840
IBM's Single 1.8 GHz G5 SPEC Scores:
SPECint 937
SPECfp 1051
According to this, Apple's own testing shows a 2.0 GHz G5 is supposedly 20% slower than the 1.8 GHz G5? Impossible? So how can you account for the difference in speeds? Say it with me...
BECAUSE APPLE RAN THEIR BENCHMARKS BY TRYING TO USE THE MOST BALANCED PLATFORM SETTINGS ON BOTH SIDES, INCLUDING THE USE OF THE GCC COMPILER, IN ORDER TO LEVEL THE PLAYING FIELD AND PROVIDE THE MOST UNBIASED SPEC SCORES THAT ARE POSSIBLE!
If Apple had used whatever optimized compiler settings that IBM ran when they tested the 1.8 GHz G5, the 2.0 GHz would have been about 10% faster than IBM's scores. That would have made them around the 1037/1151 mark, give or take. Why would Apple have published a report with scores of 800/840 instead of 1037/1151? Their SPECint scores would have been 20% higher and their SPECfp scores would have been 30% higher!
The only explanation for the discrepancy is that Apple's tests were intentionally NOT optimized...intentionally skewed down by doing exactly what they claimed: Using GCC on both sides to provide as FAIR a playing field, without tricks, as possible. And it's the fact that the PCers refuse to concede this point that is most annoying.
Apple has NOT claimed that the PC SPEC scores can't be tricked into being higher through Intel compiler optimization...that wasn't the point of THIS particular test. Do your own friggin' tests! But under THIS test, the results are valid. And Apple's claim has not yet been refuted scientifically...only attacked by the PC masses without grasping the facts. And no matter how much some may whine, there is no evidence Apple lied or intentionally misled anyone.
Apple's scores being lower than IBM's seem proof to me that Apple didn't go out of their way to falsify the test results...if they did, it hurt their own scores between 20-30%. You have to be a major conspiracy theorist to believe THAT! I do think that Apple should do new tests where they show Windows results, Intel's compiler results, etc. And then let IBM do the Apple tests with an IBM compiler. Wait til we see the crying then...
-- Ensoniq