this post was submitted on 24 Jun 2026
130 points (81.0% liked)

Selfhosted

60623 readers
932 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
 

Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

you are viewing a single comment's thread
view the rest of the comments
[–] domi@lemmy.secnd.me 12 points 2 weeks ago (1 children)

Yes, I got a Strix Halo machine before the RAM price hike and use it to run all my ML stuff on it.

Currently using llama-swap with llama.cpp/ComfyUI and opencode/Open WebUI as frontend.

I'm running Qwen3.6-27b, Voxtral Mini 4b, Piper and Qwen Image. Also, some embedding and reranking models.

I use them for:

  • Tagging and classification of my documents in Paperless
  • Home Assistant (voice assistant)
  • Translations (both text and image)
  • Transcriptions
  • Some light coding and debugging
  • Avatar/Backdrop generation for DnD sessions
[–] SuspiciousCarrot78@aussie.zone 1 points 2 weeks ago (1 children)

What sort of tok/s are you getting on the strix?

[–] domi@lemmy.secnd.me 2 points 2 weeks ago (1 children)

About 200 t/s prompt processing and 10-20 t/s with MTP.

Greatly depends on the task, predictable things like code generates at 18-20 t/s. Creative writing more like 10-17 t/s.

[–] SuspiciousCarrot78@aussie.zone 1 points 2 weeks ago

Damn - I thought strix would do a bit better than that, for how much it costs.