Technology

59673 readers

3166 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

169

The Mozilla Graveyard (www.spacebar.news)

submitted 2 months ago by corbin@infosec.pub to c/technology@lemmy.world

71 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] lime@feddit.nu 2 points 2 months ago (1 children)

llamafile is not really "effective". it's incredibly impressive, but it's the opposite of effective. it's a collection of a bunch of hacks reliant on coincidences in OS design, and works by basically recompiling itself on the fly to work with different architectures.

if you want effective, run llama.cpp compiled with actual optimizations for your platform.

[–] interdimensionalmeme@lemmy.ml 1 points 1 month ago

By effective I mean, I downloaded one file, renamed it to .exe, ran it and now my shitty company laptop was running a basic LLM with actually lots of features, right then and there. I didn't even have a GPU or admin privileges, and it just worked

Sure, maybe it's possible to squeeze more performance out of a 3 years old laptop, but that was actually very usable out of the box. And I didn't even need to unbox it !