Apple Silicon might get used for AI chips in server farms

Posted:
in Future Apple Hardware

A new rumor claims that Apple will use TSMC's 3nm technology for an AI server processor that it is designing alongside its iPhone and Mac chips.

A future iPhone with enhanced Siri



It's already known that TSMC has been developing 3 nanometer processors, and believed that Apple has bought out its entire production capacity for it. The presumption was that Apple would use the process for latest versions of its A-series iPhone chips, and M-series Mac ones.

Now, however, a leaker named "Phone Chip Expert" on Chinese social media site Weibo, claims that Apple is designing a specific, bespoke artificial intelligence processor. As first spotted by MacRumors, the leaker says it will go into mass production in the second half of 2025.

If correct, this suggests that Apple is not exclusively focusing on AI being run locally on device. It's been presumed that Apple prefers AI running on the iPhone because of privacy, but so far AI has required greater processing capacity, and storage, than is always available on device.

To that end, Apple has recently acquired firms in both Canada and France that work on compressing AI requirements. Apple has also published a research paper specifically about how it can tackle "the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters on flash memory but bringing them on demand to DRAM."

Even if, as previously expected, the iPhone will perform most of its AI on device, there may yet be more intensive processing that requires being offloaded to a data center. It makes sense for Apple to use TSMC and its technologies to make that be as fast and efficient as possible.

However, it's also the case that TSMC is an existing partner and is already producing iPhone, iPad, and Mac processors. If Apple were to work with another manufacturer, it would necessarily have to reveal its plans to them for best optimization and coordination.

The "Phone Chip Expert" does not have a current Apple rumor track record, but they do have an accurate one from about five years ago. The claim of Apple needing AI server farms and designing their processors with TSMC is plausible, and a logical assumption based on what the company will need.

Apple is expected to announce AI features as part of its iOS 18 launch at WWDC in June. At least one analyst, though, predicts that hardware-assisted AI won't be in the iPhone until 2025's iPhone 17.

Rumor Score: Possible

Read on AppleInsider

Comments

  • Reply 1 of 18
    thttht Posts: 5,512member
    This has been one of those obvious moves for long while now.

    Upcoming Nvidia server hardware is now running at up to 1000 W, requires liquid cooling and cost a whole lot of money. If Apple can produce equivalent performance at 250 to 500 W and having equivalent LLM performance at lower memory footprints, it may actually make LLM chatbots, searches, and services profitable. And they may get a two-for if that hardware can be run in a Mac Pro.

    I have not seen whether MS Co-pilot subscriptions make any profits yet. The non-MS LLM services? I think they are all operating at loss. Still early for Google to see if their LLM services will be profitable.

    Apple? All part of their ecosystem train and can be amortized across multiple products, especially if their LLM services sell hardware.
    blastdoorluke hamblyAlex1Ndanoxmattinozssfe11watto_cobrabadmonk
  • Reply 2 of 18
    avon b7avon b7 Posts: 7,774member
    tht said:
    This has been one of those obvious moves for long while now.

    Upcoming Nvidia server hardware is now running at up to 1000 W, requires liquid cooling and cost a whole lot of money. If Apple can produce equivalent performance at 250 to 500 W and having equivalent LLM performance at lower memory footprints, it may actually make LLM chatbots, searches, and services profitable. And they may get a two-for if that hardware can be run in a Mac Pro.

    I have not seen whether MS Co-pilot subscriptions make any profits yet. The non-MS LLM services? I think they are all operating at loss. Still early for Google to see if their LLM services will be profitable.

    Apple? All part of their ecosystem train and can be amortized across multiple products, especially if their LLM services sell hardware.
    Yes. If true it's been a long time coming but a welcome addition. 

    The only doubt I have is that processors themselves aren't enough. They would need a full stack solution and a lot of interconnect and software frameworks to make it all work.

    A good strategic move if they do go down that route. 
    ssfe11thtbadmonk
  • Reply 3 of 18
    9secondkox29secondkox2 Posts: 2,797member
    Hope not. Apple should go all-in with on-device. 

    Even if it takes longer, that’s fine. 

    About as long as it takes to run a photoshop action. 

    No problem. 
    libertyandfreewatto_cobra
  • Reply 4 of 18
    avon b7avon b7 Posts: 7,774member
    Hope not. Apple should go all-in with on-device. 

    Even if it takes longer, that’s fine. 

    About as long as it takes to run a photoshop action. 

    No problem. 
    Solutions need to be 'trained' and then deployed as models.

    It is possible that these servers are for some kind of training but either way, not everything can (or needs to) be done locally. 
    muthuk_vanalingamblastdoorprogrammerssfe11
  • Reply 5 of 18
    blastdoorblastdoor Posts: 3,346member
    Hope not. Apple should go all-in with on-device. 

    Even if it takes longer, that’s fine. 

    About as long as it takes to run a photoshop action. 

    No problem. 
    You can’t train an LLM on device 
    watto_cobra
  • Reply 6 of 18
    Isn’t this what the group that left to start Nuvia wanted to do at Apple and they were told that company management were not interested?
    RoderikuswilliamlondonAlex1Nwatto_cobra
  • Reply 7 of 18
    Isn’t this what the group that left to start Nuvia wanted to do at Apple and they were told that company management were not interested?

    As a typical Ccook reaction to innovation, I can imagine this will be certainly part of the story
    The guy expects to regain AI leadership by deploying former Applecar Devs and engineers.
    The guy really doesn’t understand what motivates people
    edited April 23 williamlondon
  • Reply 8 of 18
    thttht Posts: 5,512member
    Isn’t this what the group that left to start Nuvia wanted to do at Apple and they were told that company management were not interested?
    I wouldn't believe anything at face value coming from startup companies. Ultimately, 99% just want to be bought and have a nice payday. Their beliefs change in accordance to where the money comes from.

    Remember, the fate of Nuvia was being bought by Qualcomm and designing a chips that go into phones and laptops. Exactly what they were doing at Apple. Server products? Who knows if Qualcomm will do it, but they have 3 generations of phone and laptop SoCs to get into a durable cadence, and however long they are contractually required to stay at Qualcomm.

    They had a nice payday and congratulations to them. Apple? Carry apace. You can also see why Apple wouldn't touch them. Just bad business. Considering the lawsuit, Apple knew that Qualcomm hiring them was a definite possibility, if not the most probable outcome of Nuvia.
    williamlondonprogrammerAlex1Nwatto_cobra
  • Reply 9 of 18
    Apple is a sleepyhead in AI. All early birds have got their worms and Apple is slowly getting up from bed. 
    williamlondon
  • Reply 10 of 18
    XedXed Posts: 2,627member
    Apple is a sleepyhead in AI. All early birds have got their worms and Apple is slowly getting up from bed. 
    Maybe if you say it again in the next post your wish will finally come true.
    williamlondonssfe11watto_cobra
  • Reply 11 of 18
    mattinozmattinoz Posts: 2,352member
    Apple is a sleepyhead in AI. All early birds have got their worms and Apple is slowly getting up from bed. 
    If the one really profitable companies in AI are nVidia and hardware makers, and Apple are willing to regear the MacPro Chips and casing around eating their lunch. It would seem like there are a lot of worms to catch. 

    ssfe11badmonkwatto_cobra
  • Reply 12 of 18
    Apple is a sleepyhead in AI. All early birds have got their worms and Apple is slowly getting up from bed. 
    The work is just starting on deploying AI to the masses and that falls into Apple core strength.  NVDA’s AI is all about processing power and development of LLM and such whereas how it will be deployed to the masses is just getting underway.   Don’t count Apple out.  
    mattinozssfe11watto_cobra
  • Reply 13 of 18
    mattinozmattinoz Posts: 2,352member
    M4 ultra code name is what Hidra sounds like but isn't at all "the mythical multiheaded beast". Indeed it is an attractive island in Norway and Apple like naming things after attractive locations. 

    Still, the previous ultras have been multi-headed beasts. Maybe this one will be a true multi-headed beast; add, say, lots of PCIe interface with CXL support so that a MacPro or this rumoured AI server could have multiple MPX modules in a single unit. Even maybe work with other manufacturers AI accelerates (or GPU).
    watto_cobra
  • Reply 14 of 18
    danoxdanox Posts: 2,993member
    Isn’t this what the group that left to start Nuvia wanted to do at Apple and they were told that company management were not interested?
    They had a meeting with Steve Jobs (upper management) and at the time they were informed we don't do that, however Apple was deep into getting the Apple Silicon M series to market (replacing Intel) and that was a much higher priority.
    edited April 23 watto_cobra
  • Reply 15 of 18
    danoxdanox Posts: 2,993member

    Roderikus said:
    Isn’t this what the group that left to start Nuvia wanted to do at Apple and they were told that company management were not interested?

    As a typical Ccook reaction to innovation, I can imagine this will be certainly part of the story
    The guy expects to regain AI leadership by deploying former Applecar Devs and engineers.
    The guy really doesn’t understand what motivates people

    Apple has been working on many things behind the scenes when comes to Apple Silicon, Apple servers of some type are inviable just a question of when the power/wattage/performance combined with the in house Apple OS making it more and more foolish to hold back (leave money on the table). Note the same condition applies to the Mac Pro stop holding back you can't sell what you don't make.

    https://9to5mac.com/2024/03/11/apple-car-chip-four-m2-ultra/

    https://wccftech.com/apple-m2-extreme-successors-thanks-to-tsmc-3dfabric-technology/

    https://www.imore.com/mac/m3-extreme-could-succeed-where-m2-extreme-never-materialized-but-not-until-2025
    edited April 23 watto_cobra
  • Reply 16 of 18
    MarvinMarvin Posts: 15,362moderator
    Isn’t this what the group that left to start Nuvia wanted to do at Apple and they were told that company management were not interested?
    This happened around 14 years ago when Apple was in the process of discontinuing the XServe:

    https://www.theregister.com/2020/02/14/nuvia_apple_server/
    https://regmedia.co.uk/2020/02/14/williams_apple_response.pdf
    https://www.macg.co/news/voir/175852/xserve-pour-ainsi-dire-personne-ne-les-achetait-steve-jobs

    "In 2010, Williams and Keller raised this idea with Mike Culbert, their former supervisor at Apple. Culbert suggested that they put together a presentation for Steve Jobs pitching the idea of Apple building a server chip. Williams and Keller did so, and Culbert presented that opportunity to Jobs. Following the meeting, Culbert reported to Williams that Apple would not be pursuing the server chip project because Jobs was only interested in pursuing Apple’s development of consumer-focused products.

    Consistent with Jobs’ views, various senior Apple personnel, including Senior Vice President of Hardware Technologies Johny Srouji, who led the division in which Williams worked, and Tim Millet, Williams’ direct supervisor, repeatedly emphasized to Williams and, on information and belief, other Apple employees, including at a division all-hands meeting after Williams left Apple, that Apple was a consumer-focused company and not in the business of developing servers for enterprise use."

    Apple Silicon didn't arrive in a server-worthy form until Pro/Ultra M-series chips in 2021/2022. Nor was AI a major development back in 2010-2020.

    The original proposal would likely have been making chips to run Windows and Linux server OS. Mac OS is rarely used in the server environment. This isn't a huge revenue driver unless they charge a lot for the chips as the volume is low - single digit million sales per year. It only makes sense if the chips are priced at $1000+ like Xeon, Nvidia and Epyc server chips are.

    This AI setup would likely use MacOS in order to use the Neural Engine.

    Nuvia plans to make server chips with Qualcomm:

    https://arstechnica.com/gadgets/2022/08/qualcomm-is-jumping-back-into-the-server-cpu-market-with-nuvia-acquisition/

    There was a report at once point saying they'd only license the designs but it looks like they will make chips. ARM is suing them and saying their license isn't valid after the Qualcomm purchase, they have a court case in September.
    watto_cobra
  • Reply 17 of 18
    Any tech company that wants to be a player has to be a player in AI. Any company that wants to be a player in AI has to amass a lot of training & inference compute. 

    Currently king of the hill is Nvidia with H100 and upcoming B100. AMD, Grok and others are also supplying capacity and some hyperscalers like GOOG and AWS have their own silicon. Nvidia is expensive and a massive energy and resource hog.

    Apple has a fantastic silicon design team and a world class partner in TSMC and strong relationship with them. Their Neural engine, GPU and industry leading power efficiency creates interesting possibilities to explore. Since Apple also has their own dev tools, MLX frameworks, Metal etc then Nvidia's CUDA advantage may not be a factor for an in-house used Apple solution.

    Question I have is what Apple will bring to the table to network the chips together as I have yet to see Apple being strong similar to the Nvidia infiniband acquisition and NVlink.
    This is likely the deciding factor if they will prevail as large models and training requires a lot of systems to communicate very fast at scale.

    Would Apple be prudent to explore their own silicon and AI farm designs like Tesla is doing/has done with Dojo? Yes absolutely. Is Apple also likely amassing existing AI compute - of course.

    Balance of probabilities is strongly towards Apple having in-house designed server side AI compute in their labs and potentially already live in their server farms. If they get it right then their Capex story to scale with AI may become a key differentiator for earnings. Tesla got hit with a Billion dollar AI compute ramp in recent results and this has a material impact on earnings.

    edited April 24 danoxwatto_cobra
  • Reply 18 of 18
    danoxdanox Posts: 2,993member
    With in house OS/software programing combined with Apple Silicon hardware, Apple should stop leaving money on the table in this area of computing too bad it took those three engineers ending up at Qualcomm and the changing marketplace to light a fire underneath their A--.
    watto_cobra
Sign In or Register to comment.