brianus

Another weird take from the editors... don't get me wrong, this article is a public service and I'll be following its advice if and when I'm ever forced to update to iOS 18 or the latest macOS. But why focus on storage as the issue? I thought the majority of resource consumption an on-device LLM would be responsible for would be in RAM, not storage. The fact that any device that runs Apple Intelligence must have 8GB minimum is a clue to just how much you're losing when you turn this thing on. Even if it's only taking up say 3GB, that's 3GB I cannot spare on my 16GB MacBook, or even less so on an iPad, where for most models that would almost halve the available memory. Do I want webpages and apps reloading more frequently, background uploads failing and everything feeling more sluggish just so I can have an LLM of so far very questionable utility running at all times?

tiredskills said:

Why on earth would I want to run an AI model?  Locally or otherwise?

I’m sure this was meant to be snarky, but for me it’s a genuine question: what are the envisioned real world use cases? What might a business (even a home one) use a local LLM for?

The article mentions a hospital in the context of patient privacy, but what would that model actually be *doing*?

Very impressive. But is it eggshell, with Roman type? Or bone?