The hope is that Apple announces Mac Studios with M3 Max and M3 Ultra in June. If so, it's right on schedule for M4 versions in June of 2025.
There is speculation that Apple will have a discrete monolithic M3 Ultra chip with a silicon bridge to make a M3 Extreme consisting of 2 M3 Ultra chips. That would be interesting. If so, this could open up a MBP18 model or MBP16 with the M3 Ultra. Just not a lot of customers for a Mac Studio with Ultra or a Mac Pro with Ultra or Extreme.
If the next ultra is a single chip then I’ll bet it’s less than twice a max. Maybe 18 big CPU cores plus 4 little cores and 60 GPU cores.
but I could imagine Apple skipping the ultra for the m3 lineup and bringing it back for m4.
Yes, they don’t need to double. The MCM Ultra models did not scale all that well, especially GPU performance. If it is one chip, and they can maintain a nice linear core scaling, especially for the GPU cores, they won’t need to double core count at all. 60 GPU cores in a monolithic chip could outperform a MCM model with 80 GPU cores.
The issue is still size of the market. Not many people need a desktop computer with a lot of CPU cores. Then, Apple doesn’t have a presence in the gaming market which is rather significant portion of desktop GPU sales.
If they do make a monolithic Ultra chip, they should find a way to get it into more places. Put it into a MBP16 for $4000. Make it a PCIe module for the Mac Pro. Making a blade server for Mac virtualization would be a possible market, but Apple doesn’t do server hardware nor software.
I don't have any use for AI and hope its presence as a feature doesn't infect everything the way "content" has infected maps and "smart features" has infected photos. I want my apps and OS minimal, functional and linear and I want my privacy maximized. But hey that's just me.
The hope is that Apple announces Mac Studios with M3 Max and M3 Ultra in June. If so, it's right on schedule for M4 versions in June of 2025.
There is speculation that Apple will have a discrete monolithic M3 Ultra chip with a silicon bridge to make a M3 Extreme consisting of 2 M3 Ultra chips. That would be interesting. If so, this could open up a MBP18 model or MBP16 with the M3 Ultra. Just not a lot of customers for a Mac Studio with Ultra or a Mac Pro with Ultra or Extreme.
If the next ultra is a single chip then I’ll bet it’s less than twice a max. Maybe 18 big CPU cores plus 4 little cores and 60 GPU cores.
but I could imagine Apple skipping the ultra for the m3 lineup and bringing it back for m4.
Yes, they don’t need to double. The MCM Ultra models did not scale all that well, especially GPU performance. If it is one chip, and they can maintain a nice linear core scaling, especially for the GPU cores, they won’t need to double core count at all. 60 GPU cores in a monolithic chip could outperform a MCM model with 80 GPU cores.
The issue is still size of the market. Not many people need a desktop computer with a lot of CPU cores. Then, Apple doesn’t have a presence in the gaming market which is rather significant portion of desktop GPU sales.
If they do make a monolithic Ultra chip, they should find a way to get it into more places. Put it into a MBP16 for $4000. Make it a PCIe module for the Mac Pro. Making a blade server for Mac virtualization would be a possible market, but Apple doesn’t do server hardware nor software.
I think Apple should do the server thing but not to sell, to use themselves (and maybe sell indirectly through iCloud Pro branded services). In particular, use them for training AI models.
The chip fab biz is driven by economies of scale. The company that operates at the largest scale makes the most money. That used to be Intel, now it’s Apple/TSMC.
Intel lost its lead because it didn’t take mobile seriously. Apple needs to take AI seriously or it could be Nvidia that becomes TSMC’s biggest customer. Apple doesn’t want that
Apple is likely to announce M3 based updates at WWDC, or just before, for the MacStudio and Mac Pro. There will be no mention of M4. That will happen in the fall. Their update cycle is becoming clear. iPhone A18 and A18 Pro early fall (or very late summer), M4, M4 Pro, M4 Max Macbook Pros in the late fall, M4 MacBook Airs in the spring (or right before), M4 (Max, Ultra) Studios and Mac Pros Summer). Repeat cycle for A19 and M5.
Apple has indeed been delivering a Neural Processing Unit for years. However, even M3 Max is only delivering the ML performance of iPhone 14 Pro (17-18 TOPS).
The presentation on this at WWDC will be interesting. I can't find it right now, but I read somewhere there is no difference between the NPU in the A17 Pro and the M3, and the 35 trillion versus 18 trillion numbers reflect two different standards for measuring it. So one number is for the mobile industry, the other for the computer industry. ...
I'll try to find it, it wasn't an anonymous internet commenter or anything, it described the change, but it was only like two sentences and now I can't find it.
It is super interesting. I would expect the Neural Engine in the A17 Pro to be optimized for photography. Perhaps the NPU in M3 is more generic? We need benchmarks and we absolutely need for Apple to give us more AI-infused software.
Apple is likely to announce M3 based updates at WWDC, or just before, for the MacStudio and Mac Pro. There will be no mention of M4. That will happen in the fall. Their update cycle is becoming clear. iPhone A18 and A18 Pro early fall (or very late summer), M4, M4 Pro, M4 Max Macbook Pros in the late fall, M4 MacBook Airs in the spring (or right before), M4 (Max, Ultra) Studios and Mac Pros Summer). Repeat cycle for A19 and M5.
Or until Apple is ready to release said products. Let’s hope you’re right tho.
Apple has been doing ML integrations for years. Marketing-speak changed “ML” to “AI” and you’re buying into it.
As for this article, Apple’s been updating the Neural Engine every time in the silicon, sometimes obviously with the number of cores, sometimes by stating so, with giant performance increases, and other times by changing the architecture of the core.
Apple has indeed been delivering a Neural Processing Unit for years. However, even M3 Max is only delivering the ML performance of iPhone 14 Pro (17-18 TOPS). Intel is doing 34 TOPS with their 2023 laptop CPUs (combined NPU+GPU) and Lunar Lake will launch this year with 100 TOPS performance for laptops. Standing still is why the stock is dropping.
Something will need to change. I would have killed M3 Ultra and asked the entire team to work on M4 or refocused M3 Ultra on ML at any cost to show at WWDC.
The presentation on this at WWDC will be interesting. I can't find it right now, but I read somewhere there is no difference between the NPU in the A17 Pro and the M3, and the 35 trillion versus 18 trillion numbers reflect two different standards for measuring it. So one number is for the mobile industry, the other for the computer industry. Apple didn't double the peak NPU ops/sec in the A17 Pro versus the A16 Bionic. Instead, the competition changed to a different standard that provided better-sounding numbers, and Apple was forced to adopt that for the A17 Pro and the iPhone 15 Pro without comment, but they kept the old standard for the M3, to avoid appearing to claim an astounding leap forward in the NPU relative to the M1 and M2, when in reality it is incremental.
I'll try to find it, it wasn't an anonymous internet commenter or anything, it described the change, but it was only like two sentences and now I can't find it.
At 6:40, it compares M2 Max CPU, GPU and Neural Engine. The CPU gets 0.2 TOPs (using 5W), GPU gets 10.8TOPs (using 40W), NE gets 12.5TOPs (using 10W) FP16.
M2 Max GPU is 10.7TFLOPs (30-core) 13.6TFLOPs (38-core) FP32 so these numbers match and suggests the GPU isn't running FP16 twice as fast.
This means the GPU and the NE are around the same performance so if the NE is 35TOPs INT8, the higher Max GPUs are also roughly 35TOPs INT8 so total compute performance of M3 Max if both the NPU + GPU are used would be around 70TOPs. M3 Ultra would be around 100TOPs.
In practise, it can be higher or lower depending on what's being run.
Because the NE is only using 10W vs 40W GPU, they could increase the core count from 16-cores to 32-cores to get 70TOPs INT8 and potentially improve the architecture to go higher but they couldn't use the same NE across iPhone and Mac so there would have to be some kind of Neural Engine Pro/Max similar to the GPUs e.g iPhone16/Macbook M4 = 16-core NE (45TOPs), M4 Pro/Max = 32-core NE (90TOPs), M4 Ultra = 64-core NE (180TOPs).
Comments
The chip fab biz is driven by economies of scale. The company that operates at the largest scale makes the most money. That used to be Intel, now it’s Apple/TSMC.
https://www.anandtech.com/show/21116/apple-announces-m3-soc-family-m3-m3-pro-and-m3-max-make-their-marks
iPhone one likely uses INT8/FP8:
https://www.baseten.co/blog/fp8-efficient-model-inference-with-8-bit-floating-point-numbers/
Other chips like SnapDragon use INT8, Intel could be using INT4:
https://www.theregister.com/2024/04/10/hailo_10h_ai_chip/
https://www.anandtech.com/show/21346/intel-teases-lunar-lake-at-intel-vision-2024-100-tops-overall-45-tops-from-npu-alone
GPU TFLOPs are FP32.
There's a test here with machine learning FP16:
At 6:40, it compares M2 Max CPU, GPU and Neural Engine. The CPU gets 0.2 TOPs (using 5W), GPU gets 10.8TOPs (using 40W), NE gets 12.5TOPs (using 10W) FP16.
M2 Max GPU is 10.7TFLOPs (30-core) 13.6TFLOPs (38-core) FP32 so these numbers match and suggests the GPU isn't running FP16 twice as fast.
This means the GPU and the NE are around the same performance so if the NE is 35TOPs INT8, the higher Max GPUs are also roughly 35TOPs INT8 so total compute performance of M3 Max if both the NPU + GPU are used would be around 70TOPs. M3 Ultra would be around 100TOPs.
In practise, it can be higher or lower depending on what's being run.
Because the NE is only using 10W vs 40W GPU, they could increase the core count from 16-cores to 32-cores to get 70TOPs INT8 and potentially improve the architecture to go higher but they couldn't use the same NE across iPhone and Mac so there would have to be some kind of Neural Engine Pro/Max similar to the GPUs e.g iPhone16/Macbook M4 = 16-core NE (45TOPs), M4 Pro/Max = 32-core NE (90TOPs), M4 Ultra = 64-core NE (180TOPs).