Apple Silicon might get used for AI chips in server farms
A new rumor claims that Apple will use TSMC's 3nm technology for an AI server processor that it is designing alongside its iPhone and Mac chips.
It's already known that TSMC has been developing 3 nanometer processors, and believed that Apple has bought out its entire production capacity for it. The presumption was that Apple would use the process for latest versions of its A-series iPhone chips, and M-series Mac ones.
Now, however, a leaker named "Phone Chip Expert" on Chinese social media site Weibo, claims that Apple is designing a specific, bespoke artificial intelligence processor. As first spotted by MacRumors, the leaker says it will go into mass production in the second half of 2025.
If correct, this suggests that Apple is not exclusively focusing on AI being run locally on device. It's been presumed that Apple prefers AI running on the iPhone because of privacy, but so far AI has required greater processing capacity, and storage, than is always available on device.
To that end, Apple has recently acquired firms in both Canada and France that work on compressing AI requirements. Apple has also published a research paper specifically about how it can tackle "the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters on flash memory but bringing them on demand to DRAM."
Even if, as previously expected, the iPhone will perform most of its AI on device, there may yet be more intensive processing that requires being offloaded to a data center. It makes sense for Apple to use TSMC and its technologies to make that be as fast and efficient as possible.
However, it's also the case that TSMC is an existing partner and is already producing iPhone, iPad, and Mac processors. If Apple were to work with another manufacturer, it would necessarily have to reveal its plans to them for best optimization and coordination.
The "Phone Chip Expert" does not have a current Apple rumor track record, but they do have an accurate one from about five years ago. The claim of Apple needing AI server farms and designing their processors with TSMC is plausible, and a logical assumption based on what the company will need.
Apple is expected to announce AI features as part of its iOS 18 launch at WWDC in June. At least one analyst, though, predicts that hardware-assisted AI won't be in the iPhone until 2025's iPhone 17.
Rumor Score: Possible
Read on AppleInsider
Comments
Upcoming Nvidia server hardware is now running at up to 1000 W, requires liquid cooling and cost a whole lot of money. If Apple can produce equivalent performance at 250 to 500 W and having equivalent LLM performance at lower memory footprints, it may actually make LLM chatbots, searches, and services profitable. And they may get a two-for if that hardware can be run in a Mac Pro.
I have not seen whether MS Co-pilot subscriptions make any profits yet. The non-MS LLM services? I think they are all operating at loss. Still early for Google to see if their LLM services will be profitable.
Apple? All part of their ecosystem train and can be amortized across multiple products, especially if their LLM services sell hardware.
The only doubt I have is that processors themselves aren't enough. They would need a full stack solution and a lot of interconnect and software frameworks to make it all work.
A good strategic move if they do go down that route.
It is possible that these servers are for some kind of training but either way, not everything can (or needs to) be done locally.
The guy really doesn’t understand what motivates people
Remember, the fate of Nuvia was being bought by Qualcomm and designing a chips that go into phones and laptops. Exactly what they were doing at Apple. Server products? Who knows if Qualcomm will do it, but they have 3 generations of phone and laptop SoCs to get into a durable cadence, and however long they are contractually required to stay at Qualcomm.
They had a nice payday and congratulations to them. Apple? Carry apace. You can also see why Apple wouldn't touch them. Just bad business. Considering the lawsuit, Apple knew that Qualcomm hiring them was a definite possibility, if not the most probable outcome of Nuvia.
Still, the previous ultras have been multi-headed beasts. Maybe this one will be a true multi-headed beast; add, say, lots of PCIe interface with CXL support so that a MacPro or this rumoured AI server could have multiple MPX modules in a single unit. Even maybe work with other manufacturers AI accelerates (or GPU).
Apple has been working on many things behind the scenes when comes to Apple Silicon, Apple servers of some type are inviable just a question of when the power/wattage/performance combined with the in house Apple OS making it more and more foolish to hold back (leave money on the table). Note the same condition applies to the Mac Pro stop holding back you can't sell what you don't make.
https://9to5mac.com/2024/03/11/apple-car-chip-four-m2-ultra/
https://wccftech.com/apple-m2-extreme-successors-thanks-to-tsmc-3dfabric-technology/
https://www.imore.com/mac/m3-extreme-could-succeed-where-m2-extreme-never-materialized-but-not-until-2025
https://www.theregister.com/2020/02/14/nuvia_apple_server/
https://regmedia.co.uk/2020/02/14/williams_apple_response.pdf
https://www.macg.co/news/voir/175852/xserve-pour-ainsi-dire-personne-ne-les-achetait-steve-jobs
"In 2010, Williams and Keller raised this idea with Mike Culbert, their former supervisor at Apple. Culbert suggested that they put together a presentation for Steve Jobs pitching the idea of Apple building a server chip. Williams and Keller did so, and Culbert presented that opportunity to Jobs. Following the meeting, Culbert reported to Williams that Apple would not be pursuing the server chip project because Jobs was only interested in pursuing Apple’s development of consumer-focused products.
Consistent with Jobs’ views, various senior Apple personnel, including Senior Vice President of Hardware Technologies Johny Srouji, who led the division in which Williams worked, and Tim Millet, Williams’ direct supervisor, repeatedly emphasized to Williams and, on information and belief, other Apple employees, including at a division all-hands meeting after Williams left Apple, that Apple was a consumer-focused company and not in the business of developing servers for enterprise use."
Apple Silicon didn't arrive in a server-worthy form until Pro/Ultra M-series chips in 2021/2022. Nor was AI a major development back in 2010-2020.
The original proposal would likely have been making chips to run Windows and Linux server OS. Mac OS is rarely used in the server environment. This isn't a huge revenue driver unless they charge a lot for the chips as the volume is low - single digit million sales per year. It only makes sense if the chips are priced at $1000+ like Xeon, Nvidia and Epyc server chips are.
This AI setup would likely use MacOS in order to use the Neural Engine.
Nuvia plans to make server chips with Qualcomm:
https://arstechnica.com/gadgets/2022/08/qualcomm-is-jumping-back-into-the-server-cpu-market-with-nuvia-acquisition/
There was a report at once point saying they'd only license the designs but it looks like they will make chips. ARM is suing them and saying their license isn't valid after the Qualcomm purchase, they have a court case in September.
Currently king of the hill is Nvidia with H100 and upcoming B100. AMD, Grok and others are also supplying capacity and some hyperscalers like GOOG and AWS have their own silicon. Nvidia is expensive and a massive energy and resource hog.
Apple has a fantastic silicon design team and a world class partner in TSMC and strong relationship with them. Their Neural engine, GPU and industry leading power efficiency creates interesting possibilities to explore. Since Apple also has their own dev tools, MLX frameworks, Metal etc then Nvidia's CUDA advantage may not be a factor for an in-house used Apple solution.
Question I have is what Apple will bring to the table to network the chips together as I have yet to see Apple being strong similar to the Nvidia infiniband acquisition and NVlink.
This is likely the deciding factor if they will prevail as large models and training requires a lot of systems to communicate very fast at scale.
Would Apple be prudent to explore their own silicon and AI farm designs like Tesla is doing/has done with Dojo? Yes absolutely. Is Apple also likely amassing existing AI compute - of course.
Balance of probabilities is strongly towards Apple having in-house designed server side AI compute in their labs and potentially already live in their server farms. If they get it right then their Capex story to scale with AI may become a key differentiator for earnings. Tesla got hit with a Billion dollar AI compute ramp in recent results and this has a material impact on earnings.