this post was submitted on 20 Jan 2026
451 points (98.5% liked)
Fuck AI
5305 readers
1326 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Thank you, very insightful.
Really the big disguishing feature is VRAM. Us consumers just don’t have enough. If I could have a 192GB VRAM system I prolly could run a local model comparable to what OpenAI and others offer, but here I am with a lowly 12GB
You mean an Nvidia 3060? You can run GLM 4.6, a 350B model, on 12GB VRAM if you have 128GB of CPU RAM. It's not ideal though.
More practically, you can run GLM Air or Flash quite comfortably. And that'll be considerably better than "cheap" or old models like Nano, on top of being private, uncensored, and hackable/customizable.
The big distinguishing feature is "it's not for the faint of heart," heh. It takes time and tinkering to setup, as all the "easy" preconfigurations are suboptimal.
That aside, even you have a toaster, you can invest a in API credits and run open weights models with relative privacy on a self hosted front end. Pick the jurisdiction of your choosing.
For example: https://openrouter.ai/z-ai/glm-4.6v
It's like a dollar or two per million words. You can even give a middle finger to Nvidia by using Cerebras or Groq, which don't use GPUs at all.