XGrid?

macserverx

Everyone seems to have forgotten the XGrid developer mailing list that was leaked. I just now thought of it after noticing the folding in my mail account for the 21 messages I have from or regarding the list. And looking at the dates I signed up 10/12 and my last message was on 10/29. Which isn't quite as long as it seems, but the information was there. Just a thought.

barto

Well, it's CONFIRMED.



http://www.apple.com/xserve/cluster



"Xgrid, new software from Apple?s Advanced Computation Group, is one of many solutions for Mac OS X that make managing a supercomputer as easy as Macintosh."



It's linked on Apple's cluster node page, but the link goes nowhere.



Barto

barto

http://www.apple.com/pr/library/2004/jan/06xgrid.html



Apple® today previewed Xgrid?, a computational clustering technology from Apple?s Advanced Computation Group (ACG). Xgrid helps scientists and others working in compute intensive environments to fully utilize all IT resources, including desktops and servers, by creating a grid enabled ?virtual? IT environment that takes advantage of unused computing capacity to run batch and workload processing. Available as a free beta download today from www.apple.com, Xgrid brings Apple?s legendary ease-of-use to computational clustering by providing the easiest way to run compute intensive applications, such as the popular gene-sequencing application BLAST, on multiple Macs using Apple?s Rendezvous? networking technology.



Barto

jambo

http://www.apple.com/acg/xgrid/

curufinwe

If it does half of what they say it does then it is the most important thing they released today.



Tie your existing mac computer labs into a supercomputer with a plug and play solution. . .. kick ass. Gotta use that built in Gigabit ethernet for something.

jginsbu

Quote:

Originally posted by Curufinwe

If it does half of what they say it does then it is the most important thing they released today.



Tie your existing mac computer labs into a supercomputer with a plug and play solution. . .. kick ass. Gotta use that built in Gigabit ethernet for something.

I agree. And with standard office apps (of the sort people are likely to be using in labs) leaving all those Velocity Engines just sitting around with nothing to do... spare cycles anyone?

kickaha

*drool*



We have Macs scattered hither and yon in my dept.



Their cycles will be mine. Oh yes. They will be mine.

blue2kdave

this is huge...

merovingian

Quote:

Originally posted by blue2kdave

this is huge...

I totally agree.



As I've said before, clustered computing is the way of the future.



Now Apple is pushing it quite hard, (ie: investing a fair amount in it), and it should bear friut soon. m.

kickaha

Grabbed the dev kit for it.



*drool*^2.



Oh my.

macserverx

Xgrid + Xserve = LOTS of Xserve sales to come. Once Xgrid goes final anyway.



I can see Apple machines filling in the rest of the Top 10 fastest (minus EarthSim and Los Alamos) with something like this.



You say you have the dev kit. Do you, or anyone else, know if this is MVAPICH (what was ported for VA Tech)? Or anything else about what kind of standard they are using for this.

kickaha

XML layered over BEEP, apparently.

ast3r3x

What all can be clustering used for. I mean I know it isn't normally used for this, and people probably won't program this in, but could encoding DVD's be helped by clustering? I wish they made terragen capable for clustering with this. I am just curious what limits things from being used with clustering. I know like safari and other apps wouldn't even be helped because everything is instant, but can anything that takes several minutes to process be helped by clustering?

majormatt

Forgive my simplistic question but can any app utilize this?

kickaha

Apps have to have jobs that are: decomposable, require little communication between the chunks being processed. Additionally, the apps need to be able to communicate back results.



If you download the developer kit, you'll find that you write an Xgrid plugin that tells interfaces between Xgrid and your app. You then launch the batch jobs from within Xgrid, and it handles the communications between the copies of the apps on various machines.



So something like BLAST is *perfect*, (and it's gonna make my research SCREAM ) but other apps (like iDVD encoding) will have to be rewritten to take advantage of it.



Of course, if an app was designed to be externally automatable (*cough* AppleScript *cough*), then it should be a lot easier.

talksense101

Video rendering will benefit greatly from clustering. That is one common application that I can think of. Also, compiling those bloated applications from openoffice.org should be a cinch.



The success of the xgrid technology really depends on what developer tools are going to be available to the end user. Apple would do well to release the developer tool source to GNU. gcc 4.0 with a --cluster switch would be sweet (I can dream, can't I?)



P>s>: All your CPU cycles belong to us!

kickaha

Xcode already does this now.

ast3r3x

Ok so I want XGrid to be part of the OS and automatically check if the processing can be clustered and do it for you without any setup.



Would this be possible or aren't apps that dependent on the OS that the OS could do that?

kickaha

Simple. Each app in it's description of itself to the OS includes a bit about Xgriddable services and jobs, and how to break them up. Then, when the app hits a job it can break up into pieces, it asks Xgrid.framework (or whatever) to do so. The rest is taken care of.



As to what it can do *now*... I'm looking into it.

jginsbu

Here's what I find a little odd: Why the screensaver approach?



OS X seems to handle 'nice' processes with little performance lag. When I used to run the distributed.net clients, I never noticed a substantial impact on everyday, non-demanding performance. Especially since anyone who is running Xgrid of random, widely distributed desktop machines had better be doing something with low input/output needs. I would think this 'always on' approach would offer better cluster performance, because you would never waste processor time waiting 15 minutes for the screensaver to come on (that the default).



Xgrid also has a 'dedicated' mode, which is always on (doesn't wait for the screensaver). But now I wonder what the performance impact of that would be on the local user (again doing non-demanding tasks, with Xgrid running a low input/output calculation)? Is Xgrid not "niced" in this mode?

macserverx

From the look of the System Preferences Panel, you have 2 options currently:[list=1][*]Only when the computer is idle[*]Always[/list=1] Then you can choose to "Suspend tasks when computer is not idle"



The way it is described and at least how it should work, it will use unused processor cycles, just like Distributed.net does.



This isn't exactly what I had pictured. This is a fast and easy cluster setup, and not really a grid. I think of a grid as completely dynamic, without dedicated Xgrid Controller. Though it does seem as if you can be both client, controller, and agent.



Clients submit jobs, controllers dispatch jobs, and agents execute jobs. Have only been able to meddle on my PowerBook, though I'm going to try to get into the lab and put it on the two machines that meet the system requirements.



I think it'll take a little while for some more information to trickle down about what exactly is going on. I printed out the header files for the BEEP and GridPlugs frameworks, and am going to look through the Xgrid app bundle. And of course read all the included documentation.



Oh yeah, Zilla, the NeXT name for this was almost exactly like this (screen saver mode, and options). I wonder why it took them as long as it did? Though I guess they did all minds of G5 and AltiVec stuff, and added Rendezvous. But still seems like a long time for something that is almost exactly the same as Zilla.