this post was submitted on 12 Mar 2025
49 points (81.8% liked)

Selfhosted

62846 readers
614 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world
49
What AI services are you selfhosting? Or, have tested and passed on (discuss.james.network)
submitted 2 years ago by kiol@lemmy.world to c/selfhosted@lemmy.world
48 comments fedilink hide all child comments
 

Wondering about services to test on either a 16gb ram "AI Capable" arm64 board or on a laptop with modern rtx. Only looking for open source options, but curious to hear what people say. Cheers!

you are viewing a single comment's thread
view the rest of the comments
[–] Oisteink@feddit.nl 5 points 2 years ago* (last edited 2 years ago) (1 children)

I have the same setup, but its not very usable as my graphics card has 6gb ram. I want one with 20 or 24, as the 6b models are pain and the tiny ones don’t give me much.

Ollama was pretty easy to set up on windows, and its eqsy to download and test the models ollama has available

[–] kiol@lemmy.world 1 points 2 years ago (1 children)

Sounds like you and I are in a similar place of testing.

[–] Oisteink@feddit.nl 3 points 2 years ago* (last edited 2 years ago) (1 children)

Possibly. Been running it since last summer, but like i say the small models dont do much good for me. I have tried llama3.1 olmo2, deepseek r1 in a few variants, qwen2. Qwen2.5 coder, mistral, codellama, starcoder2, nemotron-mini, llama3.2, qwen2.5-coder, gamma2 and llava.

I use perplexity and mistral as paid, with much better quality. Openwebui is great though, but my hardware is lacking

Edit: saw that my mate is still using it a bit so i’ll update openwebu frpm 0.4 to 0.5.20 for him. Hes a bit anxious about sending data to the cloud so he dont mind the quality

[–] Oisteink@feddit.nl 1 points 2 years ago

Scrap that - after upgrading it went bonkers and will always use one of my «knowledges» no matter what I try. The websearch fails even with ddg as engine. Its aways seemed like the ui was made by unskilled labour, but this is just horrible. 2/10 not recommended