Xgrid lives -- a new project resurrects the promise of Apple's dead clustering software
Evoking the old Xgrid days, a new project connects Mac Studios together with Thunderbolt cables, and uses them in tandem for massively parallel computing tasks.
If you have two Mac Studios, maybe you can cluster them
A very long time ago, I was involved in cluster computing, and assisted with a few Mac-centric cluster builds in Virginia. Near the end of Xgrid availability from Apple, I also built an Xgrid cluster using beige G3 motherboards. You know, just because I could.
While the corporate- and federally-funded Xgrids were pretty good, the self-build projects were pretty janky, and fragile hacks. Apple's Xgrid worked very well in extremely specific circumstances, but very poorly outside of those scenarios.
A Mac Studio cluster really shines as the problem size . To leverage the combined GPUs, shift your thinking: choose wisely batch size & dividing of your dataset (block vs interleaving). Got 2x speedup training a large MLP on 2 MSs w/ same accuracy @awnihannun @angeloskath pic.twitter.com/P8NpPRt6TG
-- Stavros Kassinos (@KassinosS)
However, a new project called MLX that uses Macs and Thunderbolt networking appears to be much smoother than that. And even better, it uses the standard MPI distributed computing methodology.
The project installation is fairly complex, but so was Xgrid's. The new project has one master machine, and as many worker Macs as can be afforded connected directly to the master machine using Thunderbolt 4 cables. This provides extremely high-speed communications between the host machine and the workers.
The worker machines can be headless, with automatic login selected, assuming Screen Sharing is also enabled. Networking is configured manually.
Computational software is installed, using Open-MPI through HomeBrew. The MLX project repository is then installed next. Full troubleshooting and configuration are beyond the scope of this article, but that's an overview of what you need to do to get started.
As with any massively parallel calculation, scaling across devices is not quite linear. One testing cluster had three nodes working a single problem at 2.9 times faster than a single Mac Studio.
My days of configuring massively parallel systems are long over. Any work that I need to do now, with some of the folks that helped me out with the high-end professional hardware reviews that requires them now is on a pre-existing grid. They all have parameters and configuration matters decided by others.
My last work with MPI was about two decades ago with version 2, if I recall correctly. The MPI Forum members are presently evaluating versions 4.2 and 5, so my time is long past.
As such, I'm leaving most of the research and execution of this to the reader. However, a cluster of Mac Studios can be easily transported, in a package as small as a duffle bag, is power- and heat-efficient, and could use an iPad as a screen.
And, it looks easy enough to configure worker machines already nearby on adjacent desks to make an ad hoc cluster. You can connect up to six Apple Silicon Mac workers to a M1 Ultra or M2 Ultra Mac Studio host machine with Thunderbolt, if you were so inclined.
Obviously, the Mac Studio is the best choice for this from a size and power perspective. It's just perhaps not the best from a budgetary perspective.
I'll be following this project. Maybe it's time to get back into things like this.
Read on AppleInsider
Comments
Imagine this working over Wifi with any connected Apple device. A classroom where all idle CPU cycles are shared with those around you.
I know that battery life will become the central argument, but the concept is awesome!
There is almost zero chance of them selling server hardware, less than 0.1%. Same with ssh access to Private Cloud Compute, less than 0.1% chance.
Apple isn't bringing back Mac OS Server, they aren't bringing back Xserve, they aren't bringing back iPod, they aren't bringing back AirPort.
They are better off spending their R&D dollars on more wearables including (but not limited to) a consumer-priced Vision Pro HMD, perhaps a better Apple TV set-top box, perhaps better HomePod speakers.
That ship hasn’t sailed Apple is a different position than 2011 and 2013 they now have their own SOC’S which means control over their own destiny. Apple using M2 Ultra Studios to power Apple Intelligence behind the scenes (with M4, M5 versions coming in the future), was one of the biggest pieces of good news out of WWDC 24.
By using their own Apple Silicon hardware to power Apple intelligence means that the software and hardware created by Apple will be used down the road by small to medium size companies to do similar things. (And it will be used and there will be a rumbling for Apple Silicon solutions in this relatively new area of AI computing, of course it will go a long way for Apple if the solutions work for Apple as well as the what was presented at WWDC 24. Apples presentation was a Masterclass in comparison to their competition at Microsoft or Google.
The potential/current developers have taken notice of the power of Apple Silicon (large UMA memory/speed/low wattage usage) combined with modest cooling needs in comparison to their competition and that means to use it for small computer clusters, Apple using Apple Silicon servers is bad news for Apples competition, but it is good news for Apples developers and users long term.
Last week didn't go as the Tech World/Financial analysts thought it would and that is good for Apple/Apple developers and Apple users.
Apple's WWDC presentation stood head and shoulders above Microsoft Copilot+ launch.
However, that doesn't automatically mean that there are going to start shipping Apple Silicon-powered servers. My guess is that they will keep the hardware for themselves as internal only products to provide a performance-per-watt advantage over the competition. I have repeatedly speculated over the past few years that Apple was in fact testing their own custom server hardware in their datacenters. Today we know that is what they are doing.
For sure Apple has been doing their own due diligence in terms of competitive analysis. Somewhere in a lab in Cupertino (or a datacenter close by in San Jose, Fremont, etc.) there are AI accelerators from Nvidia, AMD, Intel, and others side-by-side with Apple's own hardware.
Remember that Apple really sees itself as a software company whose software and services run best on their proprietary hardware. If they wanted to be in the business of selling chips, they would have to switch to designs that are more of a compromise to deal with a wide variety of priorities from various customers (just as AMD, Intel, and NVIDIA have do). Apple doesn't have to do that with their designs: they keep what they'll use and discard what they don't want.
My guess is that Apple using their own hardware in the Private Cloud Compute will make it easier for Apple developers to write AI software that can transition between AI APIs running on local Apple Silicon hardware as well as cloud-based Apple Silicon hardware, rather than have to switch to different architecture(s) for the cloud resources.
In a way this is yet another vertical integration but rather than a consumer product stack (Mac, iPhone, iPad, Apple TV, Apple Watch) this is an Apple Intelligence resource stack.
The whole AI thing is pretty new and a lot of analysts still have a very poor grasp about the feasibility of running AI on handheld devices. Many of them have developed a pretty good understanding on the enterprise side, it's the consumer side that's still very much in its infancy.
One thing for sure, Apple has an almost unique opportunity due to their vertical product stack and tight software-hardware integration that basically no other company has. We have also seen that Microsoft's lack of any mobile presence is a major hindrance to any sort of traction they can get in today's consumer computing environment which revolves around smartphones.
But Apple will not ship Apple Silicon Xserves (or whatever they could call these boxes). At the very most they will let third-party developers eventually have some sort of API access to the cloud servers but Apple most likely will not let people just randomly run whatever AI model they want, at least not initially.
I could eventually see Apple charging developers some sort of fee for Private Cloud Compute cycles in the same way devs have to pay Amazon AWS, Microsoft Azure, Google whatever. But it will all trace back to client consumer hardware-software that has an Apple logo: iPhones, iPads, Macs, etc.
They aren't going to sell Apple Silicon server boxes so devs can deploy AI models to Android smartphones on those servers.
They have to be doing it internally right, they probably aren't building private cloud compute off stacking a bunch of Mac Studios? lol