Apple Silicon gets massive AI training speed boost with this new project

Posted:
in macOS

A new project to improve the processing speed of neural networks on Apple Silicon is potentially able to speed up training on large datasets by up to ten times.

An image generated with Image Creator from Microsoft Designer
An image generated with Image Creator from Microsoft Designer



One of the problems of creating a machine learning project is training the model on large datasets. This relies on a lot of computing power to chew through the data, but improvements here can help speed up training, and potentially improve the models.

A new project from PHD student Tristan Bilot, Francesco Farina, and the MLX team, mlx-graphs is a library intended to help Graph Neural Networks (GNNs) to run more efficiently on Apple Silicon. GNNs are used to make predictions of nodes, edges, and in performing graph-based tasks, with a particular usefulness in computer vision.

Based on MLX, the mlx-graphs project has been released as a Graph Neural Network library specifically for Apple Silicon. To researchers in the field, the project aims to provide a considerable performance boost.

Bilot claims that initial benchmarks for the library can run at up to ten times the speed of frameworks such as PyTorch Geometric and DGL when training on large graph datasets. It does so by using dedicated kernels designed to parallelize GNN computations running directly on the M-series chip's GPU.

Apple's work on Mlx-graphs is still in the early days



The project is still in its early days, having only been worked on for a few weeks, with Bilot admitting that there is "still plenty of room for major contributions." This could be a hint that more speed gains could be found with further development.

The mlx-graphs library is available on GitHub to download and install. Bilot has offered an invitation for others to explore and test the library, provide feedback, and to submit implementations via pull requests.

The project is part of a wave of interest in machine learning and generative AI, a field that can massively transform the creation of content, and the serving of information to users.

In the case of Apple, researchers within the company have created a generative AI tool for animating images. Additionally, other projects are testing the use of AI in Xcode tools.

Apple CEO Tim Cook has also spoken about the massive effort put into AI features that Apple will be rolling out to users later in 2024.



Read on AppleInsider

Comments

  • Reply 1 of 7
    blastdoorblastdoor Posts: 3,372member
    Nvidia makes crazy high profit margins on their GPUs. Apple is well positioned with their control of the “full stack” to compete, albeit indirectly, with Nvidia. 

    No, I don’t mean Apple selling GPUs. I mean Apple selling AI computation, from mobile to cloud. I’d love to see racks of M-based systems training models rather than racks of Nvidia systems. 
    lolliverdanoxwatto_cobra
  • Reply 2 of 7
    blastdoor said:
    Nvidia makes crazy high profit margins on their GPUs. Apple is well positioned with their control of the “full stack” to compete, albeit indirectly, with Nvidia. 

    No, I don’t mean Apple selling GPUs. I mean Apple selling AI computation, from mobile to cloud. I’d love to see racks of M-based systems training models rather than racks of Nvidia systems. 
    More like Apple place these training sessions and computations on each Apple silicon devices out there in the wild. You know, background crunching whilst you’re asleep 
    byronl
  • Reply 3 of 7
    mpantonempantone Posts: 2,099member
    blastdoor said:
    Nvidia makes crazy high profit margins on their GPUs. Apple is well positioned with their control of the “full stack” to compete, albeit indirectly, with Nvidia. 

    No, I don’t mean Apple selling GPUs. I mean Apple selling AI computation, from mobile to cloud. I’d love to see racks of M-based systems training models rather than racks of Nvidia systems. 
    Not going to happen.

    Apple is not in the B2B cloud service arena. If they were or wanted to be, they could have stuck a bunch of A-series or M-series SoCs in racks and sold the CPU/GPU compute power years ago. Don't forget: they ditched the Xserve, got rid of most of their high-end software offerings, and threw the OS X Server code tree into the rubbish bin.

    If Apple wants to put their silicon into AI/ML cloud computing, they will do it for their own customers: iPhone users, Mac users, Apple Watch users, Apple TV users, Apple Music subscribers, etc. Their M.O. is to deploy new technologies that benefit a wide swath of their customer base and create a competitive advantage over other companies' users.

    And where Apple would come ahead would be performance-per-watt, the metric that Johny Sroudji pounds home relentlessly. If Apple ran some sort of AI cloud facility, someone else could probably do the same with Nvidia hardware. They would just be pouring more dollars into Nvidia's wallets and feeding more dollars to the utility company. And they would still lack the seamless vertical integration that Apple can design.
    edited February 21 byronlwatto_cobra
  • Reply 4 of 7
    mattinozmattinoz Posts: 2,369member
    mpantone said:
    blastdoor said:
    Nvidia makes crazy high profit margins on their GPUs. Apple is well positioned with their control of the “full stack” to compete, albeit indirectly, with Nvidia. 

    No, I don’t mean Apple selling GPUs. I mean Apple selling AI computation, from mobile to cloud. I’d love to see racks of M-based systems training models rather than racks of Nvidia systems. 
    Not going to happen.

    Apple is not in the B2B cloud service arena. If they were or wanted to be, they could have stuck a bunch of A-series or M-series SoCs in racks and sold the CPU/GPU compute power years ago. Don't forget: they ditched the Xserve, got rid of most of their high-end software offerings, and threw the OS X Server code tree into the rubbish bin.

    If Apple wants to put their silicon into AI/ML cloud computing, they will do it for their own customers: iPhone users, Mac users, Apple Watch users, Apple TV users, Apple Music subscribers, etc. Their M.O. is to deploy new technologies that benefit a wide swath of their customer base and create a competitive advantage over other companies' users.

    And where Apple would come ahead would be performance-per-watt, the metric that Johny Sroudji pounds home relentlessly. If Apple ran some sort of AI cloud facility, someone else could probably do the same with Nvidia hardware. They would just be pouring more dollars into Nvidia's wallets and feeding more dollars to the utility company. And they would still lack the seamless vertical integration that Apple can design.
    If it helps build public-facing apps, then why wouldn't we offer this as an xCodeCloud service?
    Sure outside of maybe a single line on the developer site and 15mins at each WWDC mid week session they aren't going to make a public fuss about it, still they could make a system that pays for itself in subscriptions and benefit themselves from the 3rd people working on the optimisation.  
    byronlwatto_cobra
  • Reply 5 of 7
    blastdoorblastdoor Posts: 3,372member
    mpantone said:

    Apple is not in the B2B cloud service arena..
    Are you sure about that?

    https://developer.apple.com/xcode-cloud/
    byronlwatto_cobra
  • Reply 6 of 7
    blastdoorblastdoor Posts: 3,372member
    mattinoz said:
    mpantone said:
    blastdoor said:
    Nvidia makes crazy high profit margins on their GPUs. Apple is well positioned with their control of the “full stack” to compete, albeit indirectly, with Nvidia. 

    No, I don’t mean Apple selling GPUs. I mean Apple selling AI computation, from mobile to cloud. I’d love to see racks of M-based systems training models rather than racks of Nvidia systems. 
    Not going to happen.

    Apple is not in the B2B cloud service arena. If they were or wanted to be, they could have stuck a bunch of A-series or M-series SoCs in racks and sold the CPU/GPU compute power years ago. Don't forget: they ditched the Xserve, got rid of most of their high-end software offerings, and threw the OS X Server code tree into the rubbish bin.

    If Apple wants to put their silicon into AI/ML cloud computing, they will do it for their own customers: iPhone users, Mac users, Apple Watch users, Apple TV users, Apple Music subscribers, etc. Their M.O. is to deploy new technologies that benefit a wide swath of their customer base and create a competitive advantage over other companies' users.

    And where Apple would come ahead would be performance-per-watt, the metric that Johny Sroudji pounds home relentlessly. If Apple ran some sort of AI cloud facility, someone else could probably do the same with Nvidia hardware. They would just be pouring more dollars into Nvidia's wallets and feeding more dollars to the utility company. And they would still lack the seamless vertical integration that Apple can design.
    If it helps build public-facing apps, then why wouldn't we offer this as an xCodeCloud service?
    Sure outside of maybe a single line on the developer site and 15mins at each WWDC mid week session they aren't going to make a public fuss about it, still they could make a system that pays for itself in subscriptions and benefit themselves from the 3rd people working on the optimisation.  
    Exactly. 

    What looks like a simple call to an API from the point of view of a developer could involve massive computation in an Apple-owned facility running thousands (or more) of M-based systems. 

    The notion that because Apple killed Xserve back in 2011 does not tell us much about what Apple might do today, anymore than the cancellation of Newton told us anything about what hand-held computer Apple might one day make. 
    mattinozwatto_cobratmay
  • Reply 7 of 7
    danoxdanox Posts: 3,048member
    The UMA memory size, low wattage and overall performance of Apple Silicon computer makes them ideal/relativity affordable for this area of computing from the M1 Mac Studio Ultra 96 gig, M2 Mac Studio Ultra 192 gig and the future M3 Mac Studio Ultra 256 gig and the current MacBook Pro M3 laptop 128 gig. Many who own these particular Mac machines will start to wonder why the resale is high after 2 years.

    Apple needs to do all they behind the scenes to make it easy for those who want to use Mac's in this area of computing, by designing/engineering whatever bridge software/utilities needed, I don't think Apple can afford to let it go and this also includes
    servers down the road. (shouldn't have let those three engineer's go who ended up at Qualcomm after Nuvia was bought out).

    https://www.zdnet.com/article/how-apples-ai-advances-could-make-or-break-the-iphone-16/  Being a vertical computer company means everything.

    https://www.theregister.com/2024/01/25/apple_generative_ai/  Hmm...theregister over the years is known for being negative toward Apple but their take on AI is actually fair.

    watto_cobratmay
Sign In or Register to comment.