I had it running for image generation. As far as I recall it was slow and buggy so I didn't do more with it.
Steam Deck
A place to discuss and support all things Steam Deck.
Replacement for r/steamdeck_linux.
As Lemmy doesn't have flairs yet, you can use these prefixes to indicate what type of post you have made, eg:
[Flair] My post title
The following is a list of suggested flairs:
[Discussion] - General discussion.
[Help] - A request for help or support.
[News] - News about the deck.
[PSA] - Sharing important information.
[Game] - News / info about a game on the deck.
[Update] - An update to a previous post.
[Meta] - Discussion about this community.
Some more Steam Deck specific flairs:
[Boot Screen] - Custom boot screens/videos.
[Selling] - If you are selling your deck.
These are not enforced, but they are encouraged.
Rules:
- Follow the rules of Sopuli
- Posts must be related to the Steam Deck in an obvious way.
- No piracy, there are other communities for that.
- Discussion of emulators are allowed, but no discussion on how to illegally acquire ROMs.
- This is a place of civil discussion, no trolling.
- Have fun.
Try Vulkan in Koboldcpp https://github.com/LostRuins/koboldcpp/releases/tag/v1.81.1
I haven't heard of anyone trying it on deck but I don't see why it wouldn't work.
I probably can't help, I have ROCM acceleration working on my desktop linux PC for image gen and LLMs, but I don't think I had to install the massive rocm files for it to work.
What specifically are you trying to run using ROCM? If I remember right most of the stuff I tried just needed pytorch, which I would install in a python venv with pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.3
That's still pretty heft though, my current venv is 16GB in size, but may include some duplicate/old files from repeated upgrades without cleaning.
I'm mostly trying to run text-generation-webui for fun; what did you install to run image generation if you never installed the rocm binaries? Hopefully i should be able to replicate it in a container
Pytorch includes the necessary ROCm files for image gen as far as I know. My computer does have an /opt/ROCm folder but it's only a few megabytes in size.
Unfortunately I don't see any mention of pytorch in your linked program, and it does mention a proper ROCm install as a requirement there.
If that's the case i'm going to try to run it without installing rocm, since i saw a pytorch rocm package being downloaded by pip. I'm gonna need to reinstall though.
There are different pytorch install commands for cuda/ROCm. Usually the key to getting ROCm to work with AI GitHub projects is to edit a config file/launch option and replace the pytorch command with a ROCm one