this post was submitted on 12 Mar 2025
49 points (81.8% liked)

Selfhosted

62846 readers
693 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
 

Wondering about services to test on either a 16gb ram "AI Capable" arm64 board or on a laptop with modern rtx. Only looking for open source options, but curious to hear what people say. Cheers!

you are viewing a single comment's thread
view the rest of the comments
[–] acockworkorange@mander.xyz 1 points 2 years ago (1 children)

I am curious about trying an application specific AI. Like just for coding, for instance. I assume the memory requirements would be much lower.

[–] kiol@lemmy.world 1 points 2 years ago (1 children)

afaik Ollama would fit that bill, but perhaps others can chime in. You could probably run it on your local computer with a small model based on CPU alone.

[–] acockworkorange@mander.xyz 1 points 2 years ago

I haven’t sunk much time at it, but I’m not aware of any training data focusing on code only. There’s nothing preventing me from running with general purpose data, but I imagine I’d have a snappier response with a smaller, focused dataset, without losing accuracy.