Xgrid lives -- a new project resurrects the promise of Apple's dead clustering software

Posted:
in macOS

Evoking the old Xgrid days, a new project connects Mac Studios together with Thunderbolt cables, and uses them in tandem for massively parallel computing tasks.

Two stacked, silver Mac Studios, sitting on a white surface. Blurred background features a brick wall and a sign.
If you have two Mac Studios, maybe you can cluster them



A very long time ago, I was involved in cluster computing, and assisted with a few Mac-centric cluster builds in Virginia. Near the end of Xgrid availability from Apple, I also built an Xgrid cluster using beige G3 motherboards. You know, just because I could.

While the corporate- and federally-funded Xgrids were pretty good, the self-build projects were pretty janky, and fragile hacks. Apple's Xgrid worked very well in extremely specific circumstances, but very poorly outside of those scenarios.

A Mac Studio cluster really shines as the problem size . To leverage the combined GPUs, shift your thinking: choose wisely batch size & dividing of your dataset (block vs interleaving). Got 2x speedup training a large MLP on 2 MSs w/ same accuracy @awnihannun @angeloskath pic.twitter.com/P8NpPRt6TG

-- Stavros Kassinos (@KassinosS)



However, a new project called MLX that uses Macs and Thunderbolt networking appears to be much smoother than that. And even better, it uses the standard MPI distributed computing methodology.

The project installation is fairly complex, but so was Xgrid's. The new project has one master machine, and as many worker Macs as can be afforded connected directly to the master machine using Thunderbolt 4 cables. This provides extremely high-speed communications between the host machine and the workers.

The worker machines can be headless, with automatic login selected, assuming Screen Sharing is also enabled. Networking is configured manually.

Computational software is installed, using Open-MPI through HomeBrew. The MLX project repository is then installed next. Full troubleshooting and configuration are beyond the scope of this article, but that's an overview of what you need to do to get started.

As with any massively parallel calculation, scaling across devices is not quite linear. One testing cluster had three nodes working a single problem at 2.9 times faster than a single Mac Studio.

My days of configuring massively parallel systems are long over. Any work that I need to do now, with some of the folks that helped me out with the high-end professional hardware reviews that requires them now is on a pre-existing grid. They all have parameters and configuration matters decided by others.

My last work with MPI was about two decades ago with version 2, if I recall correctly. The MPI Forum members are presently evaluating versions 4.2 and 5, so my time is long past.

As such, I'm leaving most of the research and execution of this to the reader. However, a cluster of Mac Studios can be easily transported, in a package as small as a duffle bag, is power- and heat-efficient, and could use an iPad as a screen.

And, it looks easy enough to configure worker machines already nearby on adjacent desks to make an ad hoc cluster. You can connect up to six Apple Silicon Mac workers to a M1 Ultra or M2 Ultra Mac Studio host machine with Thunderbolt, if you were so inclined.

Obviously, the Mac Studio is the best choice for this from a size and power perspective. It's just perhaps not the best from a budgetary perspective.

I'll be following this project. Maybe it's time to get back into things like this.



Read on AppleInsider

Comments

  • Reply 1 of 13
    rezwitsrezwits Posts: 891member
    This is cool!  Always hated having to setup Xgrid, because of DNS w/Kerberos and machines had to be running the same version of Server, but was fun to tinker around with...
    watto_cobra
  • Reply 2 of 13
    blastdoorblastdoor Posts: 3,429member
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now. 


    williamlondonwatto_cobra
  • Reply 3 of 13
    Mike WuertheleMike Wuerthele Posts: 6,899administrator
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now. 


    They are. Apple's Private Cloud Compute breaks up queries that need off-device processing.
    williamlondonwatto_cobra
  • Reply 4 of 13
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now.
    Now if they'll just release the new server hardware.

    Apple hasn't had a real server-grade hardware product since 2011.  It's infuriating.

    And bring back Mac OS Server!
    tokyojimu
  • Reply 5 of 13
    blastdoorblastdoor Posts: 3,429member
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now.
    Now if they'll just release the new server hardware.

    Apple hasn't had a real server-grade hardware product since 2011.  It's infuriating.

    And bring back Mac OS Server!
    I wonder if apple will eventually provide APIs to allow developers to access Apple cloud computing resources through their apps. 

    I doubt Apple will sell server hardware, and I doubt they will have AWS like functionality in which people ssh into a server. 
    watto_cobra
  • Reply 6 of 13
    coolfactorcoolfactor Posts: 2,294member
    Very happy to see work being done in this area.

    Imagine this working over Wifi with any connected Apple device. A classroom where all idle CPU cycles are shared with those around you. 

    I know that battery life will become the central argument, but the concept is awesome! 
    watto_cobra
  • Reply 7 of 13
    mpantonempantone Posts: 2,104member
    blastdoor said:
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now.
    Now if they'll just release the new server hardware.

    Apple hasn't had a real server-grade hardware product since 2011.  It's infuriating.

    And bring back Mac OS Server!
    I wonder if apple will eventually provide APIs to allow developers to access Apple cloud computing resources through their apps. 

    I doubt Apple will sell server hardware, and I doubt they will have AWS like functionality in which people ssh into a server. 
    It is extremely unlikely that Apple would let third-party developers upload their own AI models into Apple's cloud services. However there's probably a decent chance that they will eventually (like in 2-3 years) provide some sort of API to access Apple's Private Cloud Compute resources, likely in a limited scope initially. This would be to access Apple's AI models.

    There is almost zero chance of them selling server hardware, less than 0.1%. Same with ssh access to Private Cloud Compute, less than 0.1% chance.
    edited June 17 watto_cobra
  • Reply 8 of 13
    mpantonempantone Posts: 2,104member
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now.
    Now if they'll just release the new server hardware.

    Apple hasn't had a real server-grade hardware product since 2011.  It's infuriating.

    And bring back Mac OS Server!
    That ship has sailed.

    Apple isn't bringing back Mac OS Server, they aren't bringing back Xserve, they aren't bringing back iPod, they aren't bringing back AirPort.

    They are better off spending their R&D dollars on more wearables including (but not limited to) a consumer-priced Vision Pro HMD, perhaps a better Apple TV set-top box, perhaps better HomePod speakers.
    edited June 17 williamlondonwatto_cobra
  • Reply 9 of 13
    danoxdanox Posts: 3,101member
    mpantone said:
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now.
    Now if they'll just release the new server hardware.

    Apple hasn't had a real server-grade hardware product since 2011.  It's infuriating.

    And bring back Mac OS Server!
    That ship has sailed.

    Apple isn't bringing back Mac OS Server, they aren't bringing back Xserve, they aren't bringing back iPod, they aren't bringing back AirPort.

    They are better off spending their R&D dollars on more wearables including (but not limited to) a consumer-priced Vision Pro HMD, perhaps a better Apple TV set-top box, perhaps better HomePod speakers.

    That ship hasn’t sailed Apple is a different position than 2011 and 2013 they now have their own SOC’S which means control over their own destiny. Apple using M2 Ultra Studios to power Apple Intelligence behind the scenes (with M4, M5 versions coming in the future), was one of the biggest pieces of good news out of WWDC 24. 

    By using their own Apple Silicon hardware to power Apple intelligence means that the software and hardware created by Apple will be used down the road by small to medium size companies to do similar things. (And it will be used and there will be a rumbling for Apple Silicon solutions in this relatively new area of AI computing, of course it will go a long way for Apple if the solutions work for Apple as well as the what was presented at WWDC 24. Apples presentation was a Masterclass in comparison to their competition at Microsoft or Google.

    The potential/current developers have taken notice of the power of Apple Silicon (large UMA memory/speed/low wattage usage) combined with modest cooling needs in comparison to their competition and that means to use it for small computer clusters, Apple using Apple Silicon servers is bad news for Apples competition, but it is good news for Apples developers and users long term.

    Last week didn't go as the Tech World/Financial analysts thought it would and that is good for Apple/Apple developers and Apple users.

    edited June 17 watto_cobra
  • Reply 10 of 13
    rob53rob53 Posts: 3,272member
    Ok, who's going to try this out with six Mac Studios then try running the LINPACK benchmark (https://top500.org/project/linpack/) to see what kind of numbers the benchmark gives. I'm sure the results will be faster than the original Mac "supercomputer" system of a couple decades ago and know it probably won't come anywhere near the TOP 500 but until someone tries it we won't know how fast this simple cluster actually is. TB4 only allows 6 devices on one "network" correct? Can you daisy chain more than 6 Studios? I didn't see anything on the TB5 website that you could daisy chain more than 6. This article states that you have one master Studio and can add as many more over Thunderbolt as you can afford. My question is the master Studio has six TB4 ports on it. This would mean you might/could attach six Studios to the Master. Does this also mean each Studio client could add five more Studios and keep going from there? As for a cluster computing system, the most important feature is computational power, not necessarily the speed of computer interconnections. Getting the data to each Studio might not be that big of a deal so the 10Gb Ethernet over an ethernet switch might be fast enough.
    watto_cobrarezwits
  • Reply 11 of 13
    Mike WuertheleMike Wuerthele Posts: 6,899administrator
    rob53 said:
    Ok, who's going to try this out with six Mac Studios then try running the LINPACK benchmark (https://top500.org/project/linpack/) to see what kind of numbers the benchmark gives. I'm sure the results will be faster than the original Mac "supercomputer" system of a couple decades ago and know it probably won't come anywhere near the TOP 500 but until someone tries it we won't know how fast this simple cluster actually is. TB4 only allows 6 devices on one "network" correct? Can you daisy chain more than 6 Studios? I didn't see anything on the TB5 website that you could daisy chain more than 6. This article states that you have one master Studio and can add as many more over Thunderbolt as you can afford. My question is the master Studio has six TB4 ports on it. This would mean you might/could attach six Studios to the Master. Does this also mean each Studio client could add five more Studios and keep going from there? As for a cluster computing system, the most important feature is computational power, not necessarily the speed of computer interconnections. Getting the data to each Studio might not be that big of a deal so the 10Gb Ethernet over an ethernet switch might be fast enough.
    In theory, you can use a Thunderbolt hub to connect more. How effective this is, will greatly depend on how chatty the computation that's being done is. TB4 allows six daisy-chains off of one controller, but it doesn't look like you can add downstream Thunderbolt networked worker nodes off of a worker.
    williamlondonwatto_cobrarezwits
  • Reply 12 of 13
    mpantonempantone Posts: 2,104member
    danox said:
    mpantone said:
    blastdoor said:
    Seems a safe bet that Apple is doing some large scale grid computing for Apple intelligence now.
    Now if they'll just release the new server hardware.

    Apple hasn't had a real server-grade hardware product since 2011.  It's infuriating.

    And bring back Mac OS Server!
    That ship has sailed.

    Apple isn't bringing back Mac OS Server, they aren't bringing back Xserve, they aren't bringing back iPod, they aren't bringing back AirPort.

    They are better off spending their R&D dollars on more wearables including (but not limited to) a consumer-priced Vision Pro HMD, perhaps a better Apple TV set-top box, perhaps better HomePod speakers.

    That ship hasn’t sailed Apple is a different position than 2011 and 2013 they now have their own SOC’S which means control over their own destiny. Apple using M2 Ultra Studios to power Apple Intelligence behind the scenes (with M4, M5 versions coming in the future), was one of the biggest pieces of good news out of WWDC 24. 

    By using their own Apple Silicon hardware to power Apple intelligence means that the software and hardware created by Apple will be used down the road by small to medium size companies to do similar things. (And it will be used and there will be a rumbling for Apple Silicon solutions in this relatively new area of AI computing, of course it will go a long way for Apple if the solutions work for Apple as well as the what was presented at WWDC 24. Apples presentation was a Masterclass in comparison to their competition at Microsoft or Google.

    The potential/current developers have taken notice of the power of Apple Silicon (large UMA memory/speed/low wattage usage) combined with modest cooling needs in comparison to their competition and that means to use it for small computer clusters, Apple using Apple Silicon servers is bad news for Apples competition, but it is good news for Apples developers and users long term.

    Last week didn't go as the Tech World/Financial analysts thought it would and that is good for Apple/Apple developers and Apple users.

    That ship has sailed.

    Apple's WWDC presentation stood head and shoulders above Microsoft Copilot+ launch.

    However, that doesn't automatically mean that there are going to start shipping Apple Silicon-powered servers. My guess is that they will keep the hardware for themselves as internal only products to provide a performance-per-watt advantage over the competition. I have repeatedly speculated over the past few years that Apple was in fact testing their own custom server hardware in their datacenters. Today we know that is what they are doing.

    For sure Apple has been doing their own due diligence in terms of competitive analysis. Somewhere in a lab in Cupertino (or a datacenter close by in San Jose, Fremont, etc.) there are AI accelerators from Nvidia, AMD, Intel, and others side-by-side with Apple's own hardware.

    Remember that Apple really sees itself as a software company whose software and services run best on their proprietary hardware. If they wanted to be in the business of selling chips, they would have to switch to designs that are more of a compromise to deal with a wide variety of priorities from various customers (just as AMD, Intel, and NVIDIA have do). Apple doesn't have to do that with their designs: they keep what they'll use and discard what they don't want.

    My guess is that Apple using their own hardware in the Private Cloud Compute will make it easier for Apple developers to write AI software that can transition between AI APIs running on local Apple Silicon hardware as well as cloud-based Apple Silicon hardware, rather than have to switch to different architecture(s) for the cloud resources.

    In a way this is yet another vertical integration but rather than a consumer product stack (Mac, iPhone, iPad, Apple TV, Apple Watch) this is an Apple Intelligence resource stack.

    The whole AI thing is pretty new and a lot of analysts still have a very poor grasp about the feasibility of running AI on handheld devices. Many of them have developed a pretty good understanding on the enterprise side, it's the consumer side that's still very much in its infancy.

    One thing for sure, Apple has an almost unique opportunity due to their vertical product stack and tight software-hardware integration that basically no other company has. We have also seen that Microsoft's lack of any mobile presence is a major hindrance to any sort of traction they can get in today's consumer computing environment which revolves around smartphones.

    But Apple will not ship Apple Silicon Xserves (or whatever they could call these boxes). At the very most they will let third-party developers eventually have some sort of API access to the cloud servers but Apple most likely will not let people just randomly run whatever AI model they want, at least not initially.

    I could eventually see Apple charging developers some sort of fee for Private Cloud Compute cycles in the same way devs have to pay Amazon AWS, Microsoft Azure, Google whatever. But it will all trace back to client consumer hardware-software that has an Apple logo: iPhones, iPads, Macs, etc.

    They aren't going to sell Apple Silicon server boxes so devs can deploy AI models to Android smartphones on those servers.
    muthuk_vanalingamwatto_cobrarezwits
  • Reply 13 of 13
    tipootipoo Posts: 1,149member
    With the building blocks all there, they really should make a more compute dense form factor again, they threw a rack mountable Mac Pro in there but 5U for one socket is not very dense. Bring back Xserve with Apple Silicon!

    They have to be doing it internally right, they probably aren't building private cloud compute off stacking a bunch of Mac Studios? lol 
    edited June 21 watto_cobra
Sign In or Register to comment.