this post was submitted on 03 Apr 2026
22 points (80.6% liked)

Selfhosted

58172 readers
531 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

  7. No low-effort posts. This is subjective and will largely be determined by the community member reports.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
22
Which Llama Server Hardware do you use? (discuss.tchncs.de)
submitted 1 day ago* (last edited 1 day ago) by bazinga@discuss.tchncs.de to c/selfhosted@lemmy.world
21 comments fedilink hide all child comments
 

I realize, I need to upgrade my little NUC to something bigger for higher inference of bigger llama models. I want something that you still can have on your living room's tv bench, so no monster rack please, but that has also the necessary muscle when needed for llama. Budget doesn't matter right now, want to understand what's good and what's out there. Thanks

EDIT: Wow, thanks for the inspiration, guess I need to look at bit for "how to stuff a huge graphics card into a mini box". To clarify a bit more what I want with it: I want to build a responsive personal assistant. I am dreaming of models bigger than 8B, good tool calling for things like memory, websearch etc., no coding, no image generation, no video generation required. Image recognition would be good but not a must. Regarding footprint, the no monster ;) Something that you can have in your livingroom, and could be wife approved - so no big gaming rig with exhaust pipes and stuff, needs to be good looking ;)

you are viewing a single comment's thread
view the rest of the comments
[–] RodgeGrabTheCat@sh.itjust.works 0 points 1 day ago (3 children)

From what I have observed, the Fediverse is against "AI". I doubt if you will find your answer here. "AI" is using too much water, electricity and is putting people out of work.

[–] bazinga@discuss.tchncs.de 7 points 1 day ago* (last edited 1 day ago)

I agree. I also think that there is nothing good in for-profit AI corporations. I can recommend the book "the empire of AI" However, I personally think self hosting and having full control of the use is a bit different.

[–] illusionist@lemmy.zip 7 points 1 day ago* (last edited 1 day ago)

Not the whole fediverse.

I have a good efficiency boost thanks to LLMs. They are not perfect, they lie and everything. But they write simple and good bash scripts. They know cron and regex. Stuff that I could do but I don't want to.

Creating videos is costly. My LLM usage compared to creating videos is a joke. People playing bus simulator is much worse than me asking a llm how the function is called to calculate a mean.

Ai is not the problem. People using it are.

[–] sudoMakeUser@sh.itjust.works 2 points 1 day ago

I like AI and use it often.