Selfhosted

59939 readers

285 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Self hosting LLMs on a remote VPS (lemmy.dbzer0.com)

submitted 2 years ago by EmbarrassedDrum@lemmy.dbzer0.com to c/selfhosted@lemmy.world

18 comments fedilink hide all child comments

Hi all, I'd like to hear some suggestions on self hosting LLMs on a remote server, and accessing said LLM via a client app or a convenient website. Either hear about your setups or products you got good impression on.

I've hosted Ollama before but I don't think it's intented for remote use. On the other hand I'm not really an expert and maybe there's other things to do like add-ons.

Thanks in advance!

you are viewing a single comment's thread
view the rest of the comments

[–] EmbarrassedDrum@lemmy.dbzer0.com 1 points 2 years ago (1 children)

No, but I have free instance on Oracle Cloud and that's where I'll run it. If it's too slow or no good I'll stop using it but there's no harm trying.

[–] DarkDarkHouse@lemmy.sdf.org 2 points 2 years ago (1 children)

I’d be interested to see how it goes. I’ve deployed Ollama plus Open WebUI on a few hosts and small models like Llama3.2 run adequately (at least as fast as I can read) on even an old i5-8500T with no GPU. Oracle Cloud free tier might work OK.

[–] EmbarrassedDrum@lemmy.dbzer0.com 2 points 2 years ago

then I'll let you know when I deploy it. didn't do it yet, might do it today, maybe later.