i doubt anyone is going to bring an easy way to cluster computers any time soon, but.. if anyone's going to do it, apple has the best chance of being the first to have a system that works.
because apple makes all of the hardware, they can make a standard connection (hopefully, and even possibly, GigaWire) and add it into OS X. PC companies, because it would involve the agreement of so many different companies, would take years deciding on a standard.
as i said, i doubt we'll see this any time soon, but if Apple can release it before PC companies even start attempting it, they'd probably have a decade before anyone else has something to compete with it. (unless, of course, apple decides to sell their standard for millions in royalties.. either way, apple wins!)
(as far as Anonymous Clustering being a security nightmare.. that's exactly the type of thing companies like RSA Labs need to stay in business. they'll figure out a way for it to be done. other than that, nothingtoseehere pretty much said it all)
<strong>Does anyone know how effective, if at all, clustering with Pooch is if done over the internet on say a 384 sdsl line?</strong><hr></blockquote>
Forget it. Unless your data interdependency is near zero (which hardly ever happens for scientific problems complex enough to need the combined power of many machines at all), inter-node communication is the main limiting factor in clustering. More often than not, even gigabit ethernet will be a bottleneck here, so don't even think about using SDSL...
Command line is a red herring and not worth Apple's time. Fact is most people don't need that much power. The ones who do buy fast cheap Linux boxes. It's just not worth the effort.
Well, because of the underlying BSD subsystem, you basically get the command line for free. The same most definitely cannot be said about some nifty clustering connector that's fast ($$$) enough to be worth the effort.
<strong>I know I should be looking this up myself, but I was just wondering if anyone had a good one-paragraph description of the difference between how the current Mac OS can multithread and use MP's AND how it would have to be changed to do "automatic" cluster parallel processing irregardless of software? Would it be a big change?
</strong><hr></blockquote>
Yes.
Without some additional form of (expensive) hardware support, this would add another level of indirection and incur severe speed penalties for stand-alone systems (basically, you'd at the least have to intercept every memory access the PPC does to see whether it references local or remote memory).
PC companies, because it would involve the agreement of so many different companies, would take years deciding on a standard.
as i said, i doubt we'll see this any time soon, but if Apple can release it before PC companies even start attempting it, they'd probably have a decade before anyone else has something to compete with it.
</strong><hr></blockquote>
Well, I think a decade head start might be a bit optimistic. Remember how quickly MS moved when BillG saw the internet light? Granted this is different, since it involves a communication protocol and not just an application using that protocol (so it would be more analogous to MS having to invent HTML and a browser) but still... given the massive incentive to fight back, I doubt it would take the MS/Intel axis more than 2 years, 3 at the very most, to come up with a competing solution.
But 2 or 3 years is nothing to sneeze at - a lot of Wintel users could be convinced to jump ship in that time, and most of those would stick with Apple even after clustering appeared on Wintel.
<strong>Thanks Razz, that makes sense, but why is it so "easy" for a 6th grader at U of Cal to do it?</strong><hr></blockquote>
wow, i'm really surprised that no one here seems to have any history with Appleseed clusters. I built a 4 node cluster about a year and a half ago. it was made up of 1 x G4 DP500MHz, 3 x G4 DP450MHz. the DP500 had 768MB, two of the 450's had 512, and the other had 384. all Gb ethernet, but our network admin only had a spare 10/100 switch for me to use (a good one though, Catalyst 2900). It took me about an hour to hook everything up, and install the 'Den Mother' software on all the machines. (den mother was the UC developed pred. to 'Pooch') I ran some altivec fractal test using 1,2,3,4 nodes, and it was incredible. in fact, i have screen shots somewhere, if i can find then i'll post them @mac.com.
anyway, as Gb over copper comes down in price, and people realize how incredibly easy it is to set this type of thing up, i think it will catch on a lot with universities and the scientific communities, but it doesn't (yet) have much, if any, practicle application to home use.
I perused that Slashdot thread and I have to say, as a non-techie, but science-centered person, I can sure see how intel still holds sway over the Linux faithful....what a bunch of self-aggrandizing geeks. I'm the last person to throw stones usually, but they pontificate (even more than I do) without even trying the product. It's like listening to conservative talkradio (strike that, "whineradio")... alot of vaguely logical arguments based on faulty premises, but it sounds good if it confirms your view of the world.
It of course is Apple's fault if it can't market itself better.
Most of them HATE anything not free (as beer) or not Linux. They view Macs as pretty toys without enough buzz word hardware and not cutting edge enough. They hate any company that doesn't give it away (QuickTime and Sorenson Codec). They hate any company that dictates a standard look and feel (Apple and M$). They get hung up on comparing a thrown together POS PC that they built to a high-end PowerMac ("I can build 10 PCs for the price of a couple of Powermacs").
But then again, they love their stuff like we love ours and they hate M$ so they can't be all bad.
Their biggest complaints seem to be price (there's a new one) and that Pooch wasn't complicated enough so it surely couldn't be doing anything worth looking at (nope, just a toy... move along. Nothing to see here).
Easy (but limited) Clustering is nice but I would rather have powerful, flexible and robust clustering. No that anyone should like to go diffcult for the sake of going difficult but I strikes me that the inescapble law of degrees of freedom (not to be confused with that Kevin Bacon thing). Given all the variables any given project could have there is no way to reduce complexity i.e. make it trivially easy and still maintain flexibility, tunability, etc. The more you reduce degrees of freedom i.e. control all variables to a arbitrary non-optimized value to make it "Easy" then the less optimized your setup becomes.
Its kinda like having a full gourmet kitchen and a French pastry chef baking you croissants versus some cookie dough in a Betty Crocker Easy Bake oven.
For simple non-optimized "Easy so I don't have to know anything" kind of clustering then the Pooch cluster sounds great. BUT for real i.e. optimized clustering for mission critical or heavy computations the process of computing gets rapidly complicated due to the inescapble law of degees of freedom.
One platform is not INHERENTLY "better" than another. It just depends on what you want to accomplish. I like Linux, well duh, because of its many strengths like stabilty, efficiency, and flexibility. And I personally would rather know and understand and able to control my machines rather than not have the choice.
[quote]Actually, IIRC, there were some SGI workstations which even took this one step further: They had an external connector connected to their system bus (FSB) which could be interconnected (i.e. you then had one multi-processor computer spread through two enclosures). <hr></blockquote>
At our office, our (5 year old) Origin 200 server has a CrayLink connection, resembling a buffed-up SCSI cable, connecting the two enclosures. IIRC, the throughput is 1.5 GB/sec.
That is a pretty interesting Architosh article and it brings up the inherent downside of being a software and hardware company for Apple. I sure would feel better if Apple's corporate structure gave more autonomy to its professional unit, so they could push for this kind of thing and be the conduit for UNIX migrations. That's the only way Apple came up with the first Mac, give it to a group outside the rest of the structure to innovate.
I agree with the post regarding the trade-offs of flexibility vs. simplicity and it seems obvious to me that the academic fields will always look for optimizations that Apple, MS or Dell can't spend time providing. The Linux folks will always have a job!! But the "law" of diminishing returns means that for most corporations or hobbyists, the last 5% optimization isn't worth the 20% extra effort, especially since chips improve at a faster rate than software optimization anyway.
That means that the functionality for 80% of those who cluster can probably be accomplished with software such as Pooch.
If you look at the applespeed or pooch site, I get the impression that if an application is multiple processor aware, it is cluster aware - so shouldn't everything in OSX make use of it?
</strong><hr></blockquote>
Well, in that case, you just got a wrong impression.
Seriously, applications will have to be modified (to use MacMPI, for example) and recompiled to benefit from clustering.
If you look at the applespeed or pooch site, I get the impression that if an application is multiple processor aware, it is cluster aware - so shouldn't everything in OSX make use of it?
Comments
because apple makes all of the hardware, they can make a standard connection (hopefully, and even possibly, GigaWire) and add it into OS X. PC companies, because it would involve the agreement of so many different companies, would take years deciding on a standard.
as i said, i doubt we'll see this any time soon, but if Apple can release it before PC companies even start attempting it, they'd probably have a decade before anyone else has something to compete with it. (unless, of course, apple decides to sell their standard for millions in royalties.. either way, apple wins!)
(as far as Anonymous Clustering being a security nightmare.. that's exactly the type of thing companies like RSA Labs need to stay in business. they'll figure out a way for it to be done. other than that, nothingtoseehere pretty much said it all)
[ 01-31-2002: Message edited by: confirmed ]</p>
<strong>Does anyone know how effective, if at all, clustering with Pooch is if done over the internet on say a 384 sdsl line?</strong><hr></blockquote>
Forget it. Unless your data interdependency is near zero (which hardly ever happens for scientific problems complex enough to need the combined power of many machines at all), inter-node communication is the main limiting factor in clustering. More often than not, even gigabit ethernet will be a bottleneck here, so don't even think about using SDSL...
Bye,
RazzFazz
<strong>
Riddle me this
Command line is a red herring and not worth Apple's time. Fact is most people don't need that much power. The ones who do buy fast cheap Linux boxes. It's just not worth the effort.
<img src="graemlins/bugeye.gif" border="0" alt="[Skeptical]" />
</strong><hr></blockquote>
Well, because of the underlying BSD subsystem, you basically get the command line for free. The same most definitely cannot be said about some nifty clustering connector that's fast ($$$) enough to be worth the effort.
Bye,
RazzFazz
<strong>I know I should be looking this up myself, but I was just wondering if anyone had a good one-paragraph description of the difference between how the current Mac OS can multithread and use MP's AND how it would have to be changed to do "automatic" cluster parallel processing irregardless of software? Would it be a big change?
</strong><hr></blockquote>
Yes.
Without some additional form of (expensive) hardware support, this would add another level of indirection and incur severe speed penalties for stand-alone systems (basically, you'd at the least have to intercept every memory access the PPC does to see whether it references local or remote memory).
Bye,
RazzFazz
[ 01-31-2002: Message edited by: RazzFazz ]</p>
<strong>
PC companies, because it would involve the agreement of so many different companies, would take years deciding on a standard.
as i said, i doubt we'll see this any time soon, but if Apple can release it before PC companies even start attempting it, they'd probably have a decade before anyone else has something to compete with it.
</strong><hr></blockquote>
Well, I think a decade head start might be a bit optimistic. Remember how quickly MS moved when BillG saw the internet light? Granted this is different, since it involves a communication protocol and not just an application using that protocol (so it would be more analogous to MS having to invent HTML and a browser) but still... given the massive incentive to fight back, I doubt it would take the MS/Intel axis more than 2 years, 3 at the very most, to come up with a competing solution.
But 2 or 3 years is nothing to sneeze at - a lot of Wintel users could be convinced to jump ship in that time, and most of those would stick with Apple even after clustering appeared on Wintel.
And its amazinlgy fast so umm this thread its pointless cause u can do it therefore its in current hardware
f
<strong>Thanks Razz, that makes sense, but why is it so "easy" for a 6th grader at U of Cal to do it?</strong><hr></blockquote>
wow, i'm really surprised that no one here seems to have any history with Appleseed clusters. I built a 4 node cluster about a year and a half ago. it was made up of 1 x G4 DP500MHz, 3 x G4 DP450MHz. the DP500 had 768MB, two of the 450's had 512, and the other had 384. all Gb ethernet, but our network admin only had a spare 10/100 switch for me to use (a good one though, Catalyst 2900). It took me about an hour to hook everything up, and install the 'Den Mother' software on all the machines. (den mother was the UC developed pred. to 'Pooch') I ran some altivec fractal test using 1,2,3,4 nodes, and it was incredible. in fact, i have screen shots somewhere, if i can find then i'll post them @mac.com.
anyway, as Gb over copper comes down in price, and people realize how incredibly easy it is to set this type of thing up, i think it will catch on a lot with universities and the scientific communities, but it doesn't (yet) have much, if any, practicle application to home use.
<a href="http://slashdot.org/developers/02/01/31/1757228.shtml"; target="_blank">Macintosh Clustering</a>
It of course is Apple's fault if it can't market itself better.
But then again, they love their stuff like we love ours and they hate M$ so they can't be all bad.
Their biggest complaints seem to be price (there's a new one) and that Pooch wasn't complicated enough so it surely couldn't be doing anything worth looking at (nope, just a toy... move along. Nothing to see here).
[ 01-31-2002: Message edited by: CodeWarrior ]</p>
Its kinda like having a full gourmet kitchen and a French pastry chef baking you croissants versus some cookie dough in a Betty Crocker Easy Bake oven.
For simple non-optimized "Easy so I don't have to know anything" kind of clustering then the Pooch cluster sounds great. BUT for real i.e. optimized clustering for mission critical or heavy computations the process of computing gets rapidly complicated due to the inescapble law of degees of freedom.
One platform is not INHERENTLY "better" than another. It just depends on what you want to accomplish. I like Linux, well duh, because of its many strengths like stabilty, efficiency, and flexibility. And I personally would rather know and understand and able to control my machines rather than not have the choice.
[quote]Actually, IIRC, there were some SGI workstations which even took this one step further: They had an external connector connected to their system bus (FSB) which could be interconnected (i.e. you then had one multi-processor computer spread through two enclosures). <hr></blockquote>
At our office, our (5 year old) Origin 200 server has a CrayLink connection, resembling a buffed-up SCSI cable, connecting the two enclosures. IIRC, the throughput is 1.5 GB/sec.
<a href="http://www.architosh.com/news/2002-02/2002b-0201-morepower.phtml"; target="_blank">http://www.architosh.com/news/2002-02/2002b-0201-morepower.phtml</a>;
clustering mentioned here
I agree with the post regarding the trade-offs of flexibility vs. simplicity and it seems obvious to me that the academic fields will always look for optimizations that Apple, MS or Dell can't spend time providing. The Linux folks will always have a job!! But the "law" of diminishing returns means that for most corporations or hobbyists, the last 5% optimization isn't worth the 20% extra effort, especially since chips improve at a faster rate than software optimization anyway.
That means that the functionality for 80% of those who cluster can probably be accomplished with software such as Pooch.
<strong>Hi,
If you look at the applespeed or pooch site, I get the impression that if an application is multiple processor aware, it is cluster aware - so shouldn't everything in OSX make use of it?
</strong><hr></blockquote>
Well, in that case, you just got a wrong impression.
Seriously, applications will have to be modified (to use MacMPI, for example) and recompiled to benefit from clustering.
Bye,
RazzFazz
If you look at the applespeed or pooch site, I get the impression that if an application is multiple processor aware, it is cluster aware - so shouldn't everything in OSX make use of it?
Dave