this post was submitted on 25 Apr 2024
4 points (100.0% liked)

Memes

45489 readers
788 users here now

Rules:

  1. Be civil and nice.
  2. Try not to excessively repost, as a rule of thumb, wait at least 2 months to do it if you have to.

founded 5 years ago
MODERATORS
gary_host_laptop@lemmy.ml
sexy_peach@feddit.de
cyclohexane@lemmy.ml
cypherpunks@lemmy.ml
4
Meta AI (image.nostr.build)
submitted 6 months ago by n7gifmdn@lemmy.ca to c/memes@lemmy.ml
2 comments fedilink hide all child comments
 
top 2 comments
sorted by: hot top controversial new old
[โ€“] mp3@lemmy.ca 0 points 6 months ago* (last edited 6 months ago) (1 children)

The quantized model you can run locally works decently and they can't read any of it, which is nice.

I use that one specifically https://huggingface.co/lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf

If you're looking for a relatively user-friendly software to use it, you can look at GPT4All (open source) or LM Studio.

[โ€“] passepartout@feddit.de 0 points 6 months ago

If you're ready to tinker a bit i can recommend Ollama for the backend and Open web UI for the frontend. They can also both run on the same machine.

The advantage is that you can use your GPU to compute, which is a lot faster.