blastdoor

Stabitha_Christie said:

blastdoor said:

Stabitha_Christie said:

blastdoor said:

Stabitha_Christie said:

blastdoor said:

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.

But compared to what? How much would you have to spend to do the same job on a PC?

This was actually covered in the video. You can rent a ton of server time for the cost of a maxed out Mac Studio. So if you are just using it for development or general LLM usage then it really doesn't make sense financially. That said, the maker of the video also gave examples of why it would be worth it to pay to run an LLM locally. Specially when it has to do with privacy.

My question isn’t the cost to rent but the cost to buy.

If that was the intent of your questions then your questions were really poorly worded as buying didn't come up at all.

Anyway, to answer your question, you could build a PC that could to this cheeper than you could buy a Mac Studio. The big deal about Deepseek was that it ran on consumer hardware. 

There are many versions and the 671 billion parameter version is not going to run on anything resembling a standard PC. So I think you just don't know. 

blastdoor said:

Stabitha_Christie said:

blastdoor said:

Stabitha_Christie said:

blastdoor said:

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.

But compared to what? How much would you have to spend to do the same job on a PC?

This was actually covered in the video. You can rent a ton of server time for the cost of a maxed out Mac Studio. So if you are just using it for development or general LLM usage then it really doesn't make sense financially. That said, the maker of the video also gave examples of why it would be worth it to pay to run an LLM locally. Specially when it has to do with privacy.

My question isn’t the cost to rent but the cost to buy.

If that was the intent of your questions then your questions were really poorly worded as buying didn't come up at all.

Anyway, to answer your question, you could build a PC that could to this cheeper than you could buy a Mac Studio. The big deal about Deepseek was that it ran on consumer hardware. 

There are many versions and the 671 billion parameter version is not going to run on anything resembling a standard PC. So I think you just don't know. 

So… when I said “this is covered in the video” I literally meant that is was covered in the video. All we have established here is that  you haven’t watched the video. 

Stabitha_Christie said:

blastdoor said:

Stabitha_Christie said:

blastdoor said:

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.

But compared to what? How much would you have to spend to do the same job on a PC?

This was actually covered in the video. You can rent a ton of server time for the cost of a maxed out Mac Studio. So if you are just using it for development or general LLM usage then it really doesn't make sense financially. That said, the maker of the video also gave examples of why it would be worth it to pay to run an LLM locally. Specially when it has to do with privacy.

My question isn’t the cost to rent but the cost to buy.

If that was the intent of your questions then your questions were really poorly worded as buying didn't come up at all.

Anyway, to answer your question, you could build a PC that could to this cheeper than you could buy a Mac Studio. The big deal about Deepseek was that it ran on consumer hardware. 

Stabitha_Christie said:

blastdoor said:

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.

But compared to what? How much would you have to spend to do the same job on a PC?

This was actually covered in the video. You can rent a ton of server time for the cost of a maxed out Mac Studio. So if you are just using it for development or general LLM usage then it really doesn't make sense financially. That said, the maker of the video also gave examples of why it would be worth it to pay to run an LLM locally. Specially when it has to do with privacy.

brianus said:

tiredskills said:

Why on earth would I want to run an AI model?  Locally or otherwise?

I’m sure this was meant to be snarky, but for me it’s a genuine question: what are the envisioned real world use cases? What might a business (even a home one) use a local LLM for?

The article mentions a hospital in the context of patient privacy, but what would that model actually be *doing*?

Even though the Mac Studio with M3 Ultra seems like a great option for LLM usage and development, there is a big drawback in terms of cost.