68
this post was submitted on 22 Jan 2025
68 points (98.6% liked)
AI
4786 readers
1 users here now
Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.
founded 4 years ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
There's a lot of explaining to do for Meta, OpenAI, Claude and Google gemini to justify overpaying for their models now that there's l a literal open source model that can do the basics.
I'm testing right now vscode+continue+ollama+gwen2.5-coder. With a simple GPU it's already OK.
You still need an expensive hardware to run it. Unless myceliumwebserver project will start
How much vram does your TI pack? Is that the standard 8gb ddr6?
I will because I'm surprised and impressed that a 14b model runs smoothly.
Thanks for the insights!
No worries, thank you!
i dont even have a GPU and the 14b model runs at an acceptable speed. but yes, faster and bigger would be nice.. or knowing how to distill the biggest one, cuz I only use it for something very specific.
Correct. But what's more expensive a single computing instance that's local or cloud based credit eating SAS AI that does not produce significantly better results?
Yes GPT4All of you want to try for yourself without coding know how.
The cost is a function of running an LLM at scale. You can run small models on consumer hardware, but the real contenders are using massive amounts of memory and compute on GPU arrays (plus electricity and water for cooling).
ChatGPT is reportedly losing money on their $200/mo pro subscription plan.
The same could be said for when Meta "open sourced" their models. Someone has to do the training, or else these models wouldn't exist in the first place.