Heavily upgraded M3 Ultra Mac Studio is great for AI projects
The Mac Studio is a great system for running AI models like DeepSeek locally. That is, if you're prepared to pay for M3 Ultra and a lot of upgrades.
Mac Studio
Apple Silicon is dominating the AI-capable PC market, with its capability for machine learning applications making it an attractive purchase. In a new video, it seems that the M3 Ultra Mac Studio could offer people working in AI a considerable amount of performance to play with.
The video from Dave2D discusses the M3 Ultra Mac Studio, Apple's fastest Mac on the roster. However, what is being tested in the video is more to demonstrate the extremes of the hardware, rather than a more modest and conventional loadout.
The version shown in the video uses an M3 Ultra chip with the upper-tier configuration of a 32-core CPU, an 80-core GPU, and the 32-core Neural Engine. It's also packed with 512GB of unified memory, the maximum amount available for the model, with a memory bandwidth of 819GB/s.
Local LLM usage
While the review discusses how the model doesn't make a massive change to typical video content creator workflows, it does focus on the massive amount of memory. That, and its capability to run massive Large Language Models (LLMs) used for AI applications.
The most obvious use is to be able to use an LLM locally without needing to send the request out to a server farm. In cases such as a hospital, the need for patient privacy means keeping the data on-site is a better option than sending it off-site to be processed, where possible.
To test it out, Deepseek-r1 was loaded and ran locally on the Mac Studio. This wasn't easily possible before, due to the 671-billion-parameter model requiring just over 400 gigabytes of storage and a bit less than 450 gigabytes of video RAM to function.
Since Apple Silicon uses unified memory, the top 512GB memory configuration is able to handle the massive file size of the model. Though you can run lower-parameter versions on a Mac with a smaller amount of memory, you can only really use the highest configuration to use the biggest Deepseek-r1 model.
Indeed, during testing, it was found that macOS limits the amount of memory that can be used for video memory by default to 384GB, so that had to be overridden for testing to begin.
When it was up and running, the Mac Studio was able to churn through queries at approximately 17 to 18 tokens per second. This is a level that is considered usable for the majority of LLM uses.
While LLM performance is one factor of interest, another is power consumption, as running the models can require a lot of resources. In the case of the model on the Mac Studio, it was observed as requiring 160 to 180 Watts during use.
This may sound like a lot, but it is relatively small when compared against a custom-built PC with multiple GPUs made for the same task. It's proposed that the power draw of that hypothetical system could be ten times that of the Mac Studio.
An expensive option
Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.
The 512GB option is only available for the upper-tier M3 Ultra Chip, which adds $1,500 to the base model's $3,999 price. Going from the base 96GB memory to 512GB is a $4,000 add-on again, bringing the total cost to $9,499 with the base 1TB of storage untouched.
Dropping around $10,000 on a Mac for AI purposes is out of the question for most people. But, it can be an option for businesses and corporations who have the finances and can more easily justify having a Mac Studio with M4 Ultra to run an on-site LLM.
Such operations may even consider creating a cluster of Macs, if they have the budget.
Read on AppleInsider
Comments
The article mentions a hospital in the context of patient privacy, but what would that model actually be *doing*?
This was actually covered in the video. You can rent a ton of server time for the cost of a maxed out Mac Studio. So if you are just using it for development or general LLM usage then it really doesn't make sense financially. That said, the maker of the video also gave examples of why it would be worth it to pay to run an LLM locally. Specially when it has to do with privacy.
Back in the early 1990s, Apple's IIfx was king of the Apple computer mountain. Fully loaded with 32Mb of third party memory, a very heavy 21" Radius color CRT monitor with Radius video card and a 80 MB hard drive, the cost was close to $12,000 in 1990 dollars. I checked and that amount in today's dollars is over $29,000 so the current loaded Mac M3 Ultra Mac Studio at just over $14,000 is a relative bargain.
Photoshop v1.3 was on several 3.5" discs.
One buys the tool for the job. If one is not doing the "AI" thing, then the M4 Max Mac Studio is probably sufficient or even the M4 Pro mini which is close to the power of the M1 Ultra Studio with nearly double the single core speed of the M1 Ultra.
Anyway, to answer your question, you could build a PC that could to this cheeper than you could buy a Mac Studio. The big deal about Deepseek was that it ran on consumer hardware.
The M3 Ultra is only good for those AI models. Other benchmarks confirm it is slower than the M4 Max. And of course Apple has to rip people off with $4,000 for 512GB of memory.