Is Panther really faster than Jaguar?
There has been a lot of talk about Panther being much faster than Jaguar, so I decided to do some tests with real life apps to see how widespread the speed improvements are.
All tests are on a dual G5 with 512 MB RAM and two 160 GB internal disks configured as a RAID Level 0. Each test is done 3 times on each OS to show the repeatability of results. Only the app being tested is running unless indicated otherwise.
First test is iMovie exporting a project to CD-ROM format.
10.2.8: 35.0 secs, 34.0 secs, 33.8 secs, average = 34.3 secs
10.3: 33.6 secs, 34.6 secs, 33.0 secs, average = 33.7 secs
In this test Panther is 1.7% faster.
Second test is converting a 160 MB DV movie file to MPEG2 format. I used the demo version of the BitVice MPEG2 Encoder 1.3.3, which I understand is optimized for AltiVec and multiple processors.
10.2.8: 136.1 secs, 136.3 secs, 136.5 secs, average = 136.3 secs
10.3: 133.2 secs, 133.5 secs, 133.5 secs, average = 133.4 secs
In this test Panther is 2.1% faster.
Third test is same as the second test but with two copies of SETI running in the background.
10.2.8: 140.5 secs, 140.5 secs, 140.4 secs, average = 140.5 secs
10.3: 147.4 secs, 148.4 secs, 151.3 secs, average = 149.0 secs
In this test Jaguar is 5.7% faster!
The fourth test is Gaston doing its thing on a 5000 pixel bit map.
10.2.8: 23.1 secs, 23.0 secs, 23.0 secs, average = 23.0 secs
10.3: 23.2 secs, 23.0 secs, 23.1 secs, average = 23.1 secs
In this test Jaguar and Panther are equally fast.
The common thing in all those tests is that they have a lot of number crunching activity and almost no GUI activity. As you can see, in all cases the difference between Jaguar and Panther is minimal, except in the third test where Jaguar was faster than Panther by almost 6%.
AFAIK none of the apps tested are optimized for the G5, but that is fine because the idea is to compare Jaguar against Panther, not to see how fast a G5 can ultimately be. After all, an app optimized for the G5 will run faster on Jaguar as well as in Panther.
Based on those results it seems to me the speed improvement in Panther is largely limited to the graphical user interaction, in other words, smoke and mirrors instead of real substance.
So, what do you think? Have you benchmarked any other number crunching apps and seen different results?
All tests are on a dual G5 with 512 MB RAM and two 160 GB internal disks configured as a RAID Level 0. Each test is done 3 times on each OS to show the repeatability of results. Only the app being tested is running unless indicated otherwise.
First test is iMovie exporting a project to CD-ROM format.
10.2.8: 35.0 secs, 34.0 secs, 33.8 secs, average = 34.3 secs
10.3: 33.6 secs, 34.6 secs, 33.0 secs, average = 33.7 secs
In this test Panther is 1.7% faster.
Second test is converting a 160 MB DV movie file to MPEG2 format. I used the demo version of the BitVice MPEG2 Encoder 1.3.3, which I understand is optimized for AltiVec and multiple processors.
10.2.8: 136.1 secs, 136.3 secs, 136.5 secs, average = 136.3 secs
10.3: 133.2 secs, 133.5 secs, 133.5 secs, average = 133.4 secs
In this test Panther is 2.1% faster.
Third test is same as the second test but with two copies of SETI running in the background.
10.2.8: 140.5 secs, 140.5 secs, 140.4 secs, average = 140.5 secs
10.3: 147.4 secs, 148.4 secs, 151.3 secs, average = 149.0 secs
In this test Jaguar is 5.7% faster!
The fourth test is Gaston doing its thing on a 5000 pixel bit map.
10.2.8: 23.1 secs, 23.0 secs, 23.0 secs, average = 23.0 secs
10.3: 23.2 secs, 23.0 secs, 23.1 secs, average = 23.1 secs
In this test Jaguar and Panther are equally fast.
The common thing in all those tests is that they have a lot of number crunching activity and almost no GUI activity. As you can see, in all cases the difference between Jaguar and Panther is minimal, except in the third test where Jaguar was faster than Panther by almost 6%.
AFAIK none of the apps tested are optimized for the G5, but that is fine because the idea is to compare Jaguar against Panther, not to see how fast a G5 can ultimately be. After all, an app optimized for the G5 will run faster on Jaguar as well as in Panther.
Based on those results it seems to me the speed improvement in Panther is largely limited to the graphical user interaction, in other words, smoke and mirrors instead of real substance.
So, what do you think? Have you benchmarked any other number crunching apps and seen different results?
Comments
Originally posted by Tidris
Based on those results it seems to me the speed improvement in Panther is largely limited to the graphical user interaction, in other words, smoke and mirrors instead of real substance.
To be honest:
1) I have never heard anyone claim otherwise. I also think that nobody ever believed that Panther would enable a G3 to crunch numbers like a G4, or make a G4 fly like a G5.
2) I wouldn't call the improvements in the GUI mere "smoke and mirrors". Two more years with Jaguar's Finder, and I'd have gone grey completely. An unresponsive GUI is a contradictio in terminis, except when you're working on 10.0, 10.1 or 10.2.
Originally posted by der Kopf
To be honest:
1) I have never heard anyone claim otherwise. I also think that nobody ever believed that Panther would enable a G3 to crunch numbers like a G4, or make a G4 fly like a G5.
2) I wouldn't call the improvements in the GUI mere "smoke and mirrors". Two more years with Jaguar's Finder, and I'd have gone grey completely. An unresponsive GUI is a contradictio in terminis, except when you're working on 10.0, 10.1 or 10.2.
Well, I think plenty of people expected Panther to make their current machine run much faster than Jaguar, and I think that expectation wasn't limited to the graphical interaction. That impression is based on the many Panther posts I have read on various Mac forums during the last couple of months.
As for the Jaguar Finder, I don't see how you can call it "unresponsive". I found it plenty responsive on my dual G5 and I really had no complaints about it before upgrading to Panther. Making Finder faster is a nice touch but doesn't make much difference to me.
Originally posted by Tidris
I found it plenty responsive on my dual G5 and I really had no complaints about it before upgrading to Panther.
Oh fantastic. I guess since Apple's software runs okay on the fastest, most expensive hardware available, that means it doesn't need to be optimized.
On "slightly" older machines, Panther is indeed making quite a positive difference in regards to the UI, especially in the Finder.
None of these are "smoke and mirrors," they're areas where OS X was inefficient, and it materially affected the user and the application developers.
The one possible advantage in terms of raw computational power would come from whether Panther improves Jaguar's mediocre processor-affinity algorithm. You'd need something that is designed to (thoroughly) exploit both CPUs in order to test that.
As for Finder speed, I can understand if you're not concerned from your perch on Apple's top of the line, but think of those of us on older machines.
Originally posted by Tidris
As for the Jaguar Finder, I don't see how you can call it "unresponsive". I found it plenty responsive on my dual G5 and I really had no complaints about it before upgrading to Panther. Making Finder faster is a nice touch but doesn't make much difference to me.
Plenty fast on a Dual 2.0 G5, huh? I wonder why...
Originally posted by Tidris
There has been a lot of talk about Panther being much faster than Jaguar, so I decided to do some tests with real life apps to see how widespread the speed improvements are.
All tests are on a dual G5 with 512 MB RAM and two 160 GB internal disks configured as a RAID Level 0. Each test is done 3 times on each OS to show the repeatability of results. Only the app being tested is running unless indicated otherwise.
First test is iMovie exporting a project to CD-ROM format.
10.2.8: 35.0 secs, 34.0 secs, 33.8 secs, average = 34.3 secs
10.3: 33.6 secs, 34.6 secs, 33.0 secs, average = 33.7 secs
In this test Panther is 1.7% faster.
Second test is converting a 160 MB DV movie file to MPEG2 format. I used the demo version of the BitVice MPEG2 Encoder 1.3.3, which I understand is optimized for AltiVec and multiple processors.
10.2.8: 136.1 secs, 136.3 secs, 136.5 secs, average = 136.3 secs
10.3: 133.2 secs, 133.5 secs, 133.5 secs, average = 133.4 secs
In this test Panther is 2.1% faster.
Third test is same as the second test but with two copies of SETI running in the background.
10.2.8: 140.5 secs, 140.5 secs, 140.4 secs, average = 140.5 secs
10.3: 147.4 secs, 148.4 secs, 151.3 secs, average = 149.0 secs
In this test Jaguar is 5.7% faster!
The fourth test is Gaston doing its thing on a 5000 pixel bit map.
10.2.8: 23.1 secs, 23.0 secs, 23.0 secs, average = 23.0 secs
10.3: 23.2 secs, 23.0 secs, 23.1 secs, average = 23.1 secs
In this test Jaguar and Panther are equally fast.
The common thing in all those tests is that they have a lot of number crunching activity and almost no GUI activity. As you can see, in all cases the difference between Jaguar and Panther is minimal, except in the third test where Jaguar was faster than Panther by almost 6%.
AFAIK none of the apps tested are optimized for the G5, but that is fine because the idea is to compare Jaguar against Panther, not to see how fast a G5 can ultimately be. After all, an app optimized for the G5 will run faster on Jaguar as well as in Panther.
Based on those results it seems to me the speed improvement in Panther is largely limited to the graphical user interaction, in other words, smoke and mirrors instead of real substance.
So, what do you think? Have you benchmarked any other number crunching apps and seen different results?
You G5 whore!!!!! I have an old G3. It ismuch faster!!!! I hope you rot in hell!!!! (Just kidding).
Originally posted by Amorph
The main speedups did in fact occur in places that you didn't test - OpenGL apps are faster in many cases, PDF rendering is much faster, text services are faster and scale up a lot better, animations have been sped up, etc.
I have read that the improvement bringed by panther for PDF rendering is amazing, it will be curious to see a benchmark about this one.
Originally posted by Amorph
The main speedups did in fact occur in places that you didn't test - OpenGL apps are faster in many cases, PDF rendering is much faster, text services are faster and scale up a lot better, animations have been sped up, etc.
None of these are "smoke and mirrors," they're areas where OS X was inefficient, and it materially affected the user and the application developers.
The one possible advantage in terms of raw computational power would come from whether Panther improves Jaguar's mediocre processor-affinity algorithm. You'd need something that is designed to (thoroughly) exploit both CPUs in order to test that.
As for Finder speed, I can understand if you're not concerned from your perch on Apple's top of the line, but think of those of us on older machines.
OpenGL, PDF rendering, text services, all of that is related to graphical interaction, and I already acknowledged that aspect of Panther is faster. I consider them smoke an mirrors because their main purpose is to present the results of number crunching to the user. Improvements to graphical interaction beyond a certain point is a waste of time because the human brain looking at the screen isn't fast enough to follow. A perfect example is the frame rate in games. People brag about getting 300 frames per second, but what good is that when the computer monitor refreshes at only 75 frames per second and the human brain is happy with anything over 24 frames per second?
Speaking of OpenGL, according to Xbench 1.1.3 OpenGL is slightly worse on Panter than on Jaguar, at least on my dual G5. However Quartz Graphics and User Interface tests are definitely faster.
Originally posted by Tidris
OpenGL, PDF rendering, text services, all of that is related to graphical interaction, and I already acknowledged that aspect of Panther is faster. I consider them smoke an mirrors because their main purpose is to present the results of number crunching to the user.
Oh yeah, that's got *nothing* to do with how the user interacts with the computer.
Jeez mon.
The computer without the user is a chunk of plastic. The user is the driving force for the computer "Do this, do that, get back to me". The interaction *both* directions is of prime importance to the user.
I'm sorry you have this fetish for clock cycles on number crunching (you're not a simulations engineer, are you?), and Panther didn't provide much there - it's already bloody fast for many things, and further improvements are going to have to wait until the compiler technology gets a lot more intelligent.
Algorithmic improvements almost always provide a bigger bang for the buck than oooooh-I-got-another-0.5%-squeezed-out, and that's precisely what they've done in Panther's graphics system.
Sorry it doesn't impress you. To this computer scientist/simulations engineer/software engineering researcher, it's impressive and well worth it.
Improvements to graphical interaction beyond a certain point is a waste of time ...
No kidding. And obviously we haven't reached that point yet, have we? When we do, *then* they should stop. As long as noticable changes are possible, let 'em go at it.
Originally posted by Tidris
Speaking of OpenGL, according to Xbench 1.1.3 OpenGL is slightly worse on Panter than on Jaguar, at least on my dual G5. However Quartz Graphics and User Interface tests are definitely faster.
Yeah, I noticed the same thing with Xbench on my 400Mhz G3. However, I also notice that Xbench's OGL test isn't very comprehensive or complete. It only tests one spinning squares animation inside of a window.
I find that when playing a fullscreen game, like WarCraft III, the framerate appears higher because the gameplay is smoother, but I don't have any means of testing FPS in that game.
Most people say that OpenGL games are running faster in Panther than they were in Jaguar, despite what Xbench's singular test seems to suggest.
For raw number crunching, Panther doesn't improve too much, but it does improve the experience for the end user. Text rendering, page scrolling, and general UI interaction is all improved. It's the result of numerous improvements to the OS at many different levels, certainly not "smoke and mirrors".
If you want better number crunching, hope that the applications you use are improved. Most early Mac OS X software is pretty inefficient.
I'm sure Apple will hasten the Jaguar-to-Panther migration by introducing new versions of QT, iApps, etc., that no longer work in 10.2 -- as with 10.1 and 10.0. If this is really a modern structured OS, shouldn't software and firmware work in all releases (10.0,10.1,10.2,10.3) from this century? When was the last time you looked at any software or firmware that said simply Mac OSX compatible -- 10.0 or greater?
I guess the work I do is quite GUI intensive jumping back and forth between apps/opening saving all the time. There's definately a big difference!
Originally posted by Kickaha
Oh yeah, that's got *nothing* to do with how the user interacts with the computer.
Jeez mon.
The computer without the user is a chunk of plastic. The user is the driving force for the computer "Do this, do that, get back to me". The interaction *both* directions is of prime importance to the user.
I'm sorry you have this fetish for clock cycles on number crunching (you're not a simulations engineer, are you?), and Panther didn't provide much there - it's already bloody fast for many things, and further improvements are going to have to wait until the compiler technology gets a lot more intelligent.
Algorithmic improvements almost always provide a bigger bang for the buck than oooooh-I-got-another-0.5%-squeezed-out, and that's precisely what they've done in Panther's graphics system.
Sorry it doesn't impress you. To this computer scientist/simulations engineer/software engineering researcher, it's impressive and well worth it.
No kidding. And obviously we haven't reached that point yet, have we? When we do, *then* they should stop. As long as noticable changes are possible, let 'em go at it.
For me number crunching performace is the primary concern, and the responsiveness of the machine is a secondary concern. You see, it isn't that I don't care about the GUI performace, but rather that with the hardware I am using the performance of the Jaguar GUI was already exceeding all my needs. If you had the same hardware I do perhaps you would agree, or perhaps you would not. It really is a matter of opinion after all.
You call number crunching performace a fetish, while many others call it the ability to generate revenue from their machine. The larger the number crunching capability the bigger the paycheck for a lot of Mac users out there, so please don't try to make it sound like this is a trivial issue.
Originally posted by Locomotive
When someone on this forum expresses an honest opinion, the Apple fanatics (including moderators) jump all over him (or her). Based on much Panther Beta feedback, I too was expecting 20% speed improvement across the board -- not just user interface.
I like the opposition between "honest opinion" and "fanatics". As if you couldn't arrive honestly at an opinion that didn't reflect the way things actually are. As if I haven't done all the same reading and arrived honestly at my own opinion. As if my expression was "jumping on him" instead of simply disagreeing.
I read all the lead-up too, and I know where you're coming from. I too heard about the 20% "across the board" speedups from a few testers, some of whom were talking about the combination of Panther and GCC 3. Maybe the tested apps weren't recompiled? Maybe they were, and the speedup just didn't pan out as reported? These are all possible, and they don't require some myth of oppression by the Powers That Be in order to make sense. Believe me, if we wanted this place to be the MacNet2 forums (where, at least for a long time, criticism of Apple was in violation of the user agreement!), we have the power to make it so. But we don't want that.
By the way, I'm still reading accounts of "whole new computer" speedups from people with the release version of Panther. Maybe they're mostly interacting with the parts of the OS that Apple really sped up, but that could be because Apple was really smart about deciding which parts of the OS to speed up. It is, in fact, possible to have an honest opinion that acknowledges that Apple can do right by its users every so often.
I'm sure Apple will hasten the Jaguar-to-Panther migration by introducing new versions of QT, iApps, etc., that no longer work in 10.2 -- as with 10.1 and 10.0. If this is really a modern structured OS, shouldn't software and firmware work in all releases (10.0,10.1,10.2,10.3) from this century? When was the last time you looked at any software or firmware that said simply Mac OSX compatible -- 10.0 or greater?
That would really be nice, but the sad fact is that 10.0 and 10.1 were missing large chunks of basic functionality (for example, there was no way for a printer driver to specify edge-to-edge). CoreAudio didn't appear until 10.1, and it was broken until 10.2, and the brushed metal theme didn't appear as part of the OS until 10.2. Text support lagged all the way until Panther, which is one reason why the word processor landscape on OS X only recently started to get interesting. You could call Panther the first really complete release of OS X. Nevertheless, Apple is supporting 10.1 in many of its iApps (since it's a free upgrade from 10.0, it's not that big a deal that they aren't supporting 10.0).
On the other hand, given that Apple had no credibility shipping a next-generation OS, they had to ship whatever they had on March 28, 2001. If they'd waited until now it's a fair bet that Apple would be dead. At the very least, OS X let them move forward, software- and hardware-wise.
So yes, I agree, it sucks. I'm fully aware of the fact that the hopes I had for OS X three years ago are only now beginning to be realized, because it was from its mother's womb untimely ripped. It also sucks that Apple has been charging $129 a year for us all to watch it mature. But I'm not sure that Apple could have done better, given their circumstances.
Originally posted by Tidris
For me number crunching performace is the primary concern, and the responsiveness of the machine is a secondary concern. You see, it isn't that I don't care about the GUI performace, but rather that with the hardware I am using the performance of the Jaguar GUI was already exceeding all my needs. If you had the same hardware I do perhaps you would agree, or perhaps you would not. It really is a matter of opinion after all.
You call number crunching performace a fetish, while many others call it the ability to generate revenue from their machine. The larger the number crunching capability the bigger the paycheck for a lot of Mac users out there, so please don't try to make it sound like this is a trivial issue.
I remember an old thread where people said exactly the contrary : PC users won't switch to the mac due to the disapointing GUI performance.
Panther is faster than Jaguar, and it's impressive. it's not so common that a new OS is faster than the older one. Generally it's just the contrary.
I understand that you where disapointed, too many over optimistic people claimed that Panther will improve the speed of everything. The truth is different, but the improvement is quite real for the GUI. Panther is not a miracle, but it worth his bucks.
It's not "faster", it's "as fast as Jaguar -ought- to have been" had Apple bothered to fix the bug they introduced into 10.2.8.
Lack of 30 second pauses, less beachballs, quick application launches doesn't mean "faster" it means "acceptable". Jaguar was "unacceptable".
Originally posted by johnq
Panther has no pre-binding bug, 10.2.8 does. This is what makes Panther seem fast.
It's not "faster", it's "as fast as Jaguar -ought- to have been" had Apple bothered to fix the bug they introduced into 10.2.8.
Lack of 30 second pauses, less beachballs, quick application launches doesn't mean "faster" it means "acceptable". Jaguar was "unacceptable".
What's a pre-binding bug?
Originally posted by Messiah
What's a pre-binding bug?
10.2.8 introduced a bug in which fix_prebinding or redo_prebinding (I forget which now) would run before any application launch - including background processes. This is why looking up a webpage, or if you misspelled a word, or any number of other basic tasks would give a 30 second pause. A look at top -u showed one of the prebinding processes monopolizing the CPU around 80-100%.
Not just my machine. I suspect it's very common, but most people don't bother looking for the cause, they just attribute it to Jaguar being slow, which, in and of itself it is not slow.
Originally posted by Tidris
For me number crunching performace is the primary concern, and the responsiveness of the machine is a secondary concern. You see, it isn't that I don't care about the GUI performace, but rather that with the hardware I am using the performance of the Jaguar GUI was already exceeding all my needs. If you had the same hardware I do perhaps you would agree, or perhaps you would not. It really is a matter of opinion after all.
You call number crunching performace a fetish, while many others call it the ability to generate revenue from their machine. The larger the number crunching capability the bigger the paycheck for a lot of Mac users out there, so please don't try to make it sound like this is a trivial issue.
You show a basic misunderstanding here of what an OS does.
The OS does not "crunch numbers", it merely allows a program which crunches numbers to run. Looked at from this standpoint, your performance differences show potentially huge efficiency gains at the OS level as OS code is such a small portion of the sample size when "crunching numbers". In reality, most of your numbers are just showing what is most likely just plain old variation. Small percentage efficiency returns will only show large gains when you force changes and invoked interrupts into the number crunching such as when you need i/o for the crunches.
Summarizing this, a good OS stays out of the way when it doesn't need to get in the way. 10.2.8 was already good at that, 10.3 seems not to have taken a backwards step there. Beyond that your test methodologies don't really show anything significant as they are very variable. Especially the SETI test, that one is completely invalid unless you went to extreme lengths to ensure everything was identical.
Everyone also needs to beware of the snake-oil salesman and over-exuberance which leads directly to misplaced expectations. Most of the Apple hype during WWDC centered around increased OS performance and radically better GUI performance. Somewhat misinformed folks and short memories on the subtle details lead quickly to over-generalizations which then morph into outright inaccurate statements which lead to over enthusiastic expectations and later abject disappointment as reality catches up. **Notice the difference in stating OS performance, and across the board performance. Several rumor sites went with the latter when they only had information on the former**
With the appropriate hindsight applied--behind the scenes OS code was said to have as much as 20% overall speed gains from new compilers and other optimizations. That 20% doesn't make everything 20% faster, it just gets the OS out of the way 20% faster. When the OS is staying out of the way like it should those 20% optimizations don't add up to much real time. With thing like GUI driven code, most of what you see is OS performance so the speedups become obvious.
People need to base their need/desire/budget on where in the continuum from pure computation (little OS speed difference) to heavy I/O & GUI use (lots of OS difference) they lie.