blastdoor

Stabitha_Christie said:

blastdoor said:

Stabitha_Christie said:

blastdoor said:

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.

But compared to what? How much would you have to spend to do the same job on a PC?

This was actually covered in the video. You can rent a ton of server time for the cost of a maxed out Mac Studio. So if you are just using it for development or general LLM usage then it really doesn't make sense financially. That said, the maker of the video also gave examples of why it would be worth it to pay to run an LLM locally. Specially when it has to do with privacy.

My question isn’t the cost to rent but the cost to buy.

If that was the intent of your questions then your questions were really poorly worded as buying didn't come up at all.

Anyway, to answer your question, you could build a PC that could to this cheeper than you could buy a Mac Studio. The big deal about Deepseek was that it ran on consumer hardware. 

There are many versions and the 671 billion parameter version is not going to run on anything resembling a standard PC. So I think you just don't know. 

brianus said:

tiredskills said:

Why on earth would I want to run an AI model?  Locally or otherwise?

I’m sure this was meant to be snarky, but for me it’s a genuine question: what are the envisioned real world use cases? What might a business (even a home one) use a local LLM for?

The article mentions a hospital in the context of patient privacy, but what would that model actually be *doing*?

Anybody who still doesn’t see the value of AI does not have a bright future career wise. But maybe the people still making these comments are retired, in which case I’m jealous — I’ve got a lot of years left.

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.

My thinking on Tim Cook in 2021:

https://forums.appleinsider.com/discussion/comment/3330877/#Comment_3330877

Back then I said the most important next step for him -- after many accomplishments-- was to pick a successor. 

I don't think he should be fired over this AI thing. But after the Car debacle and now this AI thing, I think it's become increasingly clear that he really does need to work on identifying a successor and then retiring. He's done a good job, but it's close to time for him to go. 

Would have been nice to get M4 Ultra, but the Studio with the base M3 Ultra looks pretty appealing to me. 

I work for a private firm that has lost a lot of revenue due to arbitrary DOGE contract cancellations, so my job is currently at risk. But if I can hold onto my job, I might spring for that base Ultra.