<strong>Unfortunately the system can't just "intercept computationally expensive instructions" or "relocate some threads". Its not particular instructions that make a task expensive, its the number of instructions and how many times they execute. The relocation of threads isn't possible because threads are coded to assume a shared memory model -- something which doesn't exist in a clustered situation.</strong><hr></blockquote>
AFAIK, VMS clustering presents a single memory space to running applications.
But then, there's a reason why VMS costs so much, and only runs on relatively high-end 64 bit hardware.
Not sure who people think will use clustering. At UCB, Stanford, MIT, etc. you typically put a proposal together for a given funding agency (DOE, NSF, Private group, etc). In the proposal their is an equipment budget to get specialized equipment for the project (Cray computers -in the old days-, HP electronic scanners, DMM's, etc any equipment needed). The agency funds the project and you get the equipment. In these lean times a lot of the funding agencies don't like to put money out on capital equipment (computers and such) because they don't want to fund multi-year projects and only fund on a year by year basis. The abilty to buy 10 G4 computers that can be justified (and paid for by) for different uses is a BIG plus in getting the funding and university approval. Not only that, you can do part of the work on a couple of G4's and get a better chance of funding by having a better grip on the research aspects of the project (all by clusteruing a couple of Mac's together). This would be a big deal to some of the groups I work with.
<strong>For those that have extra macs not doing much will not find this whole idea dumb.
Just because U of M won't use the tech does not mean its worthless.
</strong><hr></blockquote>
Hum. Never heard of a school which as a bunch of macs (which could run OS X) just 'laying around'.
BTW, I totally agree with scott_h_phd. At, MY university it is the same way.
Folks who NEED clusters, BUY clusters.
(and for programmers sake....)
Folks who NEED clusters, and can't afford clusters, BUY a newspaper and start hunting for another line of work.
Apple developing 'magic software' which lets us share our processors, and ram etc with others won't subvert the cluster market (because pros will keep buying clusters) and it will only cause endless problems with those other folks who want to milk all the new g4s which are just 'laying around' for all their worth. Our networking department would never allow such ruckus on campus as this. <img src="graemlins/oyvey.gif" border="0" alt="[No]" />
[sorry, had to get back in the tread, I knew this would be a runaway from the begining]
[ 06-24-2002: Message edited by: Jonathan Brisby ]</p>
<strong>Folks who NEED clusters, BUY clusters.</strong><hr></blockquote>
I think you need to amend this to read: "Folks who NEED clusters and can afford them, BUY clusters." Those that can't afford them don't buy them and make-do without.
I also don't understand what this whole argument is about -- what good is a cluster if you don't have software that supports clustering? If Apple provides a technology that makes "clusterable" software much easier to write, then all of a sudden clusters are much more useful. Whether or not anybody is going to cobble clusters together out of spare computers is irrelevant to the creation of such a technology by Apple.
<strong>Hum. Never heard of a school which as a bunch of macs (which could run OS X) just 'laying around'.
</strong><hr></blockquote>
Over here at UCLA, professors in the English department buy the newest Superdrive G4s, just because they can. They're barely comfortable running Microsoft Word, but they figure since they only have to pay 10% of the price, why not go for it?
A bunch of Macs sitting around? No, not really, I'm sure they're being used to type up assignments occasionally. Lots of gigaflops available? Hell yes.
Now, for those folks that think clustering technology is not 'needed', please realize that just because your imagination and ingenuity is lacking that does not deter the rest of us from using our computing resources a bunch better than you (if given the tools).
As for who needs it? I could use it! Presently I have (at home) 4 PC-Linux boxes under my desk + my cube + my laptop (not counting the three other computers for my daughter,son and wife). I'd replace these things with Macs if they had the magic clustering technology described at the top of this thread. I am "just a coder", but throughout my career I was (and am) always in need of more processing power. So, here I am in need of processing power and 5-8 of my PCs are doing absolutely nothing (relatively) to get my current job done faster! At my customer sites there are hundreds of machines yet none of them are available to me for the projects that would clearly benefit those customer.
I tend to buy leading edge hardware that lasts me several years (which I recycle as Unix boxes for some specific purpose) so to me personally clustering (as described) would be a HUGE win.
Now, if I can buy Macs and drop the clustering software on them I can keep these things as long as the hardware lasts (or software compatibility is maintained). For example, I was contemplating the purchase of a new Mac and was wondering what to do with my existing one... Enter clustering and the old machine will sit around doing what it can to make my "user experience" better. And I would buy the new hardware because it is now relatively CHEAPER since I am able to use the old machine's processing power as well.
As far as "professional" environments are concerned at work we are considering Macs as the next developer platform (instead of Sun workstations coupled with PCs). We are in need of simulating (for testing purposes) dozens of our proprietary distributed software systems. Will we go out and buy a ready made cluster? NO! Will we go out and buy 10-20 Macs for starters? I think so! Especially, if this clustering deal is real.
I'm sitting here after hours and there are... let's see 140 cubicles x 1 Sun workstation x 1 PC + a half dozen Sun servers ... (~300 boxes). Incidentally it takes me 2 hours to run a test session on my very own workstation (just basic stuff). Would I be able to sell management on the idea that Macs are the way to go and we get a cluster to boot we could actually use? More than likely.
On the other hand if you use your Mac to surf the web and post to this board and not much else (ie: word processing) then you don't need clustering. Nor do you _need_ a new Mac for that matter!
Clustering is a given in terms of the future of computing. The question is whether Apple can bring this to the masses cost effectively and in the near future.
ok I have had enough of this. Scott, john, you guys are so full of crap you don't even realize it. The reason i am so upset is because IF we had something like that at my school, we could have had an awesome video vs. a barely put together pos. And now i will explain.
I live in Clear Lake. Easily a high middle class area. The School District however is poor as **** .
At my HIGHSCHOOL there are around 5 maclabs with 20-30 G4's. one of the labs has dual 450's. I work in the journalism class. I am The station manager, tech director, lead editor, etc. I run the News Group and if has to deal with a computer then I am usualy the one fixing it or doing it. We dont get te nice G4's we have an old 350. Thats right a 350 with 512 MB of RAM. With Block Scheduling etc. Do you honestly think i would need more computers if i hadaround 100 not doing anything. This number does not include the G4 or G3 that every teacher has hooked up to the network. I could max out the network before i even began to use all the proccessing pwer of the school, let alone the district because we are all hooked up via fibre optics. Now that you know the hardware ill show you the problem. My little team had to produce a video that had to be just as good as a professional's because the district could afford anymore for their little project. Shattered Dreams was a recreation of a drunk driving incident at school. We Film as being real and its supposed to make everyone cry etc. we had one day to shoot, and edit the enitre thing. The longgest part came not from editing, deciding how we wanted things to look, or even going out for pizza at 3 am. but waiting for our dinky little g4 to render transistions, layers etc. If We had "wolf" then life would have beedn so much easier. especially with all the thing we try. you say only pro's would want it and they can afford clusters. i am telling you from expierence that kids use and max computers everyday trying to learn this stuff. and we need something fast. I am no longer at Clear Lake but the kids there want to try blue screens and etc next year, and when i tried that at home it took forever to render. so trust me at public schools "wolf" would be a god send.
p.s. they still owe me 300$ for that 80GB HD i bought them.
Wolf to me sounds more like an X server type of a thing. I kind of doubt that it would be put into the next revision of X for the imac/emac. A wolf cluster would probably be implemented on xserves.
Maybe it's just me but a cluster of emacs just seems kind of stupid. This kind of tech in an imac seems like it would really harm hardware sales. It sounds cool, but apple isn't going to shoot themselves in the foot.
p.s. allenmcjones is probably full of crap anyway.
Can't you just download the software "pooch" to do stuff like this? And for the people that are naysayers to the "grid" computing model, it's been all over the PC magazines lately. Of course, as usual with the PC mags, the uses are all theoretical and overstated as the "next big thing" since "the internet." Reading the articles makes me ill and also makes me realize why Dvorak can keep a column in a magazine like this: the editors are too dumb to realize that he's dumb.
<strong>ok I have had enough of this. Scott, john, you guys are so full of crap you don't even realize it. The reason i am so upset is because IF we had something like that at my school, we could have had an awesome video vs. a barely put together pos...</strong><hr></blockquote>
Exactly! This is one of many examples of how such a technology could be beneficial to people and to Apple's market share.
For those saying that there is no need for such a thing, I think that you are looking at clustering purely within the confines of how they are used currently. Why are they only being used in those cases now? Because it's prohibitively expensive to justify for a limited use. Now, however, you are talking about the cost being only marginally higher (the cost of Macs over POS Dells).
I'll give you another example of where this would be useful. Chemistry research labs... There are often around of a half-dozen computers in a given university research lab (in Canada. US average is much higher) for use by grad students, etc. Some are hooked up to instruments that are only used sometimes. Others are used for report/paper writing and email, internet/literature searches. And then in synthetic labs, there is often one considerably better machine used for solving X-ray crystalography structures (can take many hours). If "Wolf" existed, you could be damn sure that many of those labs would switch all those machines to Macs (and probably pretty high-end ones) so that not only can those machines be doing there intended purpose but also can contribute to solving that structure in a fraction of the time it takes currently. Right now, however, no Chem. prof can justify (or get funding for) a cluster dedicated to X-ray crystalography, because it's not used all the time and the reality is that they can wait half a day to get the answer. Would they like to not have to wait that long? You bet your A$$ they would!
For those saying that such a technology would harm Apple's hardware sales: nonsense. This kind of clustering is useful only for accelerating certain kinds of computations, otherwise the overhead of chopping it up and shipping it across a network just isn't worth it. Most every day computing won't benefit from clustering, and it is this "normal" computing that will continue to drive Apple sales. To then be able to have your software reach out beyond the limits of your desktop or notebook Mac and easily harness large (or huge) amounts of computing power for the tasks which need it will just make having a Mac even more desireable.
<strong>For those saying that such a technology would harm Apple's hardware sales: nonsense. This kind of clustering is useful only for accelerating certain kinds of computations, otherwise the overhead of chopping it up and shipping it across a network just isn't worth it. Most every day computing won't benefit from clustering, and it is this "normal" computing that will continue to drive Apple sales. To then be able to have your software reach out beyond the limits of your desktop or notebook Mac and easily harness large (or huge) amounts of computing power for the tasks which need it will just make having a Mac even more desireable.</strong><hr></blockquote>
Yeah, people will still upgrade their machines to speed up Aqua v. 2 (complete with holographic 3D Dock!) which will see no benefit whatsoever from clustering!
I am not sure why people are arguing so strongly against this idea. The rumour could be 100% bogus, but as a concept, it has it's merits.
As has been mentioned, centres that require gigantic amounts of processing power, will usually go with custom solutions if they can afford them. As has also been mentioned, 'Wolf', as a concept, would allow people who need, or simply could use clustering abilities, to implement a relatively cheap and MUCH easier solution than what is currently available.
Scientific research is so obviously an area Apple wants into, it isn't funny. Same with Video production. In these two areas alone, there are many groups/individuals/companies that do not have the funding or technical expertise to implement a custom clustering solution. Yet, one can not argue that some/many/most of these groups would jump at the opportunity to harness the power of a cluster in an easy and relatively cheap manner.
Imagine being a small animation company. Unless you can manage to find 100's of millions of dollars in funding, how can you possibly compete with powerhouses like Pixar, ILM and Disney? You could have the greatest talent in the game, yet you would have no chance. They would be able to deliver fasr faster than you could hope to. When you consider the compute power required to generate a minute of high end animation, clustering becomes a necessity, not an option. If Apple provided the ability to use your existing hardware to suddenly give you exponetially more horsepower, wouldn't you jump at this? You could add more Macs as your business grows, and each addition adds to your pool of cpu's. This could make big waves for Apple in this field. The same would hold true for smaller orgs doing scientific research, whether in genome decoding and sequencing, physics, geology, etc.
Anyway, the point of this now long winded post, is that there wouldn't really be a downside for Apple here. They would be guaranteed to improve their reputation in just these two fields, and there are others that would benefit as well. The Appleseed project is the best example of diy clustering and the benefits. I can't see a downsided to making this easier for people to do on their own. And it can only help Apple's image.
Once again this is what I've been saying the fscking time. If there is a cluster solution then THEY WILL BUY A CLUSTER. NOT look for a way to run on other people's hardware.</strong><hr></blockquote>
Your frustration is misdirected. I quoted that article not to jump into the cluster/collaboration debate but to respond to your assertion that people who want clusters don't/won't buy Macs.
Custom clusters are limited to specific tasks and applications. By having that cluster ability in every OSX system this will be the most common cluster of all. And so, the software companies will have a reason for optimizing there applications for this cluster. Times are changing, just because the companies didnt use macclusters in the past doesnt meen its always going to be like this.
Comments
<strong>Unfortunately the system can't just "intercept computationally expensive instructions" or "relocate some threads". Its not particular instructions that make a task expensive, its the number of instructions and how many times they execute. The relocation of threads isn't possible because threads are coded to assume a shared memory model -- something which doesn't exist in a clustered situation.</strong><hr></blockquote>
AFAIK, VMS clustering presents a single memory space to running applications.
But then, there's a reason why VMS costs so much, and only runs on relatively high-end 64 bit hardware.
<strong>AFAIK, VMS clustering presents a single memory space to running applications.
But then, there's a reason why VMS costs so much, and only runs on relatively high-end 64 bit hardware.
Yikes... I have a hard time imagining that being very fast unless apps are coded to avoid thrashing memory pages.
<strong>I think programmer said it best in this or another thread when he said you can't lump all schools and how they handle tech in one bunch.
Some will find a use for this tech others will buy new machines. For those that have extra macs not doing much will not find this whole idea dumb.
Just because U of M won't use the tech does not mean its worthless.
<img src="graemlins/oyvey.gif" border="0" alt="[No]" /> </strong><hr></blockquote>
Not sure who people think will use clustering. At UCB, Stanford, MIT, etc. you typically put a proposal together for a given funding agency (DOE, NSF, Private group, etc). In the proposal their is an equipment budget to get specialized equipment for the project (Cray computers -in the old days-, HP electronic scanners, DMM's, etc any equipment needed). The agency funds the project and you get the equipment. In these lean times a lot of the funding agencies don't like to put money out on capital equipment (computers and such) because they don't want to fund multi-year projects and only fund on a year by year basis. The abilty to buy 10 G4 computers that can be justified (and paid for by) for different uses is a BIG plus in getting the funding and university approval. Not only that, you can do part of the work on a couple of G4's and get a better chance of funding by having a better grip on the research aspects of the project (all by clusteruing a couple of Mac's together). This would be a big deal to some of the groups I work with.
<strong>For those that have extra macs not doing much will not find this whole idea dumb.
Just because U of M won't use the tech does not mean its worthless.
</strong><hr></blockquote>
Hum. Never heard of a school which as a bunch of macs (which could run OS X) just 'laying around'.
BTW, I totally agree with scott_h_phd. At, MY university it is the same way.
Folks who NEED clusters, BUY clusters.
(and for programmers sake....)
Folks who NEED clusters, and can't afford clusters, BUY a newspaper and start hunting for another line of work.
Apple developing 'magic software' which lets us share our processors, and ram etc with others won't subvert the cluster market (because pros will keep buying clusters) and it will only cause endless problems with those other folks who want to milk all the new g4s which are just 'laying around' for all their worth. Our networking department would never allow such ruckus on campus as this. <img src="graemlins/oyvey.gif" border="0" alt="[No]" />
[sorry, had to get back in the tread, I knew this would be a runaway from the begining]
[ 06-24-2002: Message edited by: Jonathan Brisby ]</p>
<strong>Folks who NEED clusters, BUY clusters.</strong><hr></blockquote>
I think you need to amend this to read: "Folks who NEED clusters and can afford them, BUY clusters." Those that can't afford them don't buy them and make-do without.
I also don't understand what this whole argument is about -- what good is a cluster if you don't have software that supports clustering? If Apple provides a technology that makes "clusterable" software much easier to write, then all of a sudden clusters are much more useful. Whether or not anybody is going to cobble clusters together out of spare computers is irrelevant to the creation of such a technology by Apple.
<strong>Hum. Never heard of a school which as a bunch of macs (which could run OS X) just 'laying around'.
</strong><hr></blockquote>
Over here at UCLA, professors in the English department buy the newest Superdrive G4s, just because they can. They're barely comfortable running Microsoft Word, but they figure since they only have to pay 10% of the price, why not go for it?
A bunch of Macs sitting around? No, not really, I'm sure they're being used to type up assignments occasionally. Lots of gigaflops available? Hell yes.
Ever hear of a bunch of macs sitting in companies outside of schools? That's what i am talking about.
Now, for those folks that think clustering technology is not 'needed', please realize that just because your imagination and ingenuity is lacking that does not deter the rest of us from using our computing resources a bunch better than you (if given the tools).
As for who needs it? I could use it! Presently I have (at home) 4 PC-Linux boxes under my desk + my cube + my laptop (not counting the three other computers for my daughter,son and wife). I'd replace these things with Macs if they had the magic clustering technology described at the top of this thread. I am "just a coder", but throughout my career I was (and am) always in need of more processing power. So, here I am in need of processing power and 5-8 of my PCs are doing absolutely nothing (relatively) to get my current job done faster! At my customer sites there are hundreds of machines yet none of them are available to me for the projects that would clearly benefit those customer.
I tend to buy leading edge hardware that lasts me several years (which I recycle as Unix boxes for some specific purpose) so to me personally clustering (as described) would be a HUGE win.
Now, if I can buy Macs and drop the clustering software on them I can keep these things as long as the hardware lasts (or software compatibility is maintained). For example, I was contemplating the purchase of a new Mac and was wondering what to do with my existing one... Enter clustering and the old machine will sit around doing what it can to make my "user experience" better. And I would buy the new hardware because it is now relatively CHEAPER since I am able to use the old machine's processing power as well.
As far as "professional" environments are concerned at work we are considering Macs as the next developer platform (instead of Sun workstations coupled with PCs). We are in need of simulating (for testing purposes) dozens of our proprietary distributed software systems. Will we go out and buy a ready made cluster? NO! Will we go out and buy 10-20 Macs for starters? I think so! Especially, if this clustering deal is real.
I'm sitting here after hours and there are... let's see 140 cubicles x 1 Sun workstation x 1 PC + a half dozen Sun servers ... (~300 boxes). Incidentally it takes me 2 hours to run a test session on my very own workstation (just basic stuff). Would I be able to sell management on the idea that Macs are the way to go and we get a cluster to boot we could actually use? More than likely.
On the other hand if you use your Mac to surf the web and post to this board and not much else (ie: word processing) then you don't need clustering. Nor do you _need_ a new Mac for that matter!
Clustering is a given in terms of the future of computing. The question is whether Apple can bring this to the masses cost effectively and in the near future.
I sure bloody hope so!
[ 06-24-2002: Message edited by: iCode ]</p>
I live in Clear Lake. Easily a high middle class area. The School District however is poor as **** .
At my HIGHSCHOOL there are around 5 maclabs with 20-30 G4's. one of the labs has dual 450's. I work in the journalism class. I am The station manager, tech director, lead editor, etc. I run the News Group and if has to deal with a computer then I am usualy the one fixing it or doing it. We dont get te nice G4's we have an old 350. Thats right a 350 with 512 MB of RAM. With Block Scheduling etc. Do you honestly think i would need more computers if i hadaround 100 not doing anything. This number does not include the G4 or G3 that every teacher has hooked up to the network. I could max out the network before i even began to use all the proccessing pwer of the school, let alone the district because we are all hooked up via fibre optics. Now that you know the hardware ill show you the problem. My little team had to produce a video that had to be just as good as a professional's because the district could afford anymore for their little project. Shattered Dreams was a recreation of a drunk driving incident at school. We Film as being real and its supposed to make everyone cry etc. we had one day to shoot, and edit the enitre thing. The longgest part came not from editing, deciding how we wanted things to look, or even going out for pizza at 3 am. but waiting for our dinky little g4 to render transistions, layers etc. If We had "wolf" then life would have beedn so much easier. especially with all the thing we try. you say only pro's would want it and they can afford clusters. i am telling you from expierence that kids use and max computers everyday trying to learn this stuff. and we need something fast. I am no longer at Clear Lake but the kids there want to try blue screens and etc next year, and when i tried that at home it took forever to render. so trust me at public schools "wolf" would be a god send.
p.s. they still owe me 300$ for that 80GB HD i bought them.
Maybe it's just me but a cluster of emacs just seems kind of stupid. This kind of tech in an imac seems like it would really harm hardware sales. It sounds cool, but apple isn't going to shoot themselves in the foot.
p.s. allenmcjones is probably full of crap anyway.
Matthew
<strong>ok I have had enough of this. Scott, john, you guys are so full of crap you don't even realize it. The reason i am so upset is because IF we had something like that at my school, we could have had an awesome video vs. a barely put together pos...</strong><hr></blockquote>
Exactly! This is one of many examples of how such a technology could be beneficial to people and to Apple's market share.
For those saying that there is no need for such a thing, I think that you are looking at clustering purely within the confines of how they are used currently. Why are they only being used in those cases now? Because it's prohibitively expensive to justify for a limited use. Now, however, you are talking about the cost being only marginally higher (the cost of Macs over POS Dells).
I'll give you another example of where this would be useful. Chemistry research labs... There are often around of a half-dozen computers in a given university research lab (in Canada. US average is much higher) for use by grad students, etc. Some are hooked up to instruments that are only used sometimes. Others are used for report/paper writing and email, internet/literature searches. And then in synthetic labs, there is often one considerably better machine used for solving X-ray crystalography structures (can take many hours). If "Wolf" existed, you could be damn sure that many of those labs would switch all those machines to Macs (and probably pretty high-end ones) so that not only can those machines be doing there intended purpose but also can contribute to solving that structure in a fraction of the time it takes currently. Right now, however, no Chem. prof can justify (or get funding for) a cluster dedicated to X-ray crystalography, because it's not used all the time and the reality is that they can wait half a day to get the answer. Would they like to not have to wait that long? You bet your A$$ they would!
<strong>For those saying that such a technology would harm Apple's hardware sales: nonsense. This kind of clustering is useful only for accelerating certain kinds of computations, otherwise the overhead of chopping it up and shipping it across a network just isn't worth it. Most every day computing won't benefit from clustering, and it is this "normal" computing that will continue to drive Apple sales. To then be able to have your software reach out beyond the limits of your desktop or notebook Mac and easily harness large (or huge) amounts of computing power for the tasks which need it will just make having a Mac even more desireable.</strong><hr></blockquote>
Yeah, people will still upgrade their machines to speed up Aqua v. 2 (complete with holographic 3D Dock!) which will see no benefit whatsoever from clustering!
As has been mentioned, centres that require gigantic amounts of processing power, will usually go with custom solutions if they can afford them. As has also been mentioned, 'Wolf', as a concept, would allow people who need, or simply could use clustering abilities, to implement a relatively cheap and MUCH easier solution than what is currently available.
Scientific research is so obviously an area Apple wants into, it isn't funny. Same with Video production. In these two areas alone, there are many groups/individuals/companies that do not have the funding or technical expertise to implement a custom clustering solution. Yet, one can not argue that some/many/most of these groups would jump at the opportunity to harness the power of a cluster in an easy and relatively cheap manner.
Imagine being a small animation company. Unless you can manage to find 100's of millions of dollars in funding, how can you possibly compete with powerhouses like Pixar, ILM and Disney? You could have the greatest talent in the game, yet you would have no chance. They would be able to deliver fasr faster than you could hope to. When you consider the compute power required to generate a minute of high end animation, clustering becomes a necessity, not an option. If Apple provided the ability to use your existing hardware to suddenly give you exponetially more horsepower, wouldn't you jump at this? You could add more Macs as your business grows, and each addition adds to your pool of cpu's. This could make big waves for Apple in this field. The same would hold true for smaller orgs doing scientific research, whether in genome decoding and sequencing, physics, geology, etc.
Anyway, the point of this now long winded post, is that there wouldn't really be a downside for Apple here. They would be guaranteed to improve their reputation in just these two fields, and there are others that would benefit as well. The Appleseed project is the best example of diy clustering and the benefits. I can't see a downsided to making this easier for people to do on their own. And it can only help Apple's image.
<strong>
Once again this is what I've been saying the fscking time. If there is a cluster solution then THEY WILL BUY A CLUSTER. NOT look for a way to run on other people's hardware.</strong><hr></blockquote>
Your frustration is misdirected. I quoted that article not to jump into the cluster/collaboration debate but to respond to your assertion that people who want clusters don't/won't buy Macs.
Custom clusters are limited to specific tasks and applications. By having that cluster ability in every OSX system this will be the most common cluster of all. And so, the software companies will have a reason for optimizing there applications for this cluster. Times are changing, just because the companies didnt use macclusters in the past doesnt meen its always going to be like this.