tskwara

About

Username
tskwara
Joined
Visits
11
Last Active
Roles
member
Points
16
Badges
0
Posts
12

Reactions

12Like1Dislike1Informative

  • Run ChatGPT-style AI on your Mac with OpenAI's new offline tools

    tskwaratskwara
    mariowinco said:
    tskwara said:
    blastdoor said:
    An m3 ultra Mac Studio could be configured with enough RAM to run the 120 billion parameter model. I wonder how it would perform 
    It’s running really well on my M1 Ultra 20-core CPU, a 64-core GPU, and 128GB of unified memory.  A little speechless ATM.
    Can I ask what you do with that LLM (that cannot be done online)?
    Saving some expense by running less demanding AI tasks (i.e. sythesizing datasets for SME model tuning) locally is one thing.  It can be done online but does cost more.
    williamlondonmuthuk_vanalingam