I had it running for image generation. As far as I recall it was slow and buggy so I didn't do more with it.
Steam Hardware
A place to discuss and support all Steam Hardware, including Steam Deck, Steam Machine, Steam Frame, and SteamOS in general.
As Lemmy doesn't have flairs yet, you can use these prefixes to indicate what type of post you have made, eg:
[Flair] My post title
The following is a list of suggested flairs:
[Deck] - Steam Deck related.
[Machine] - Steam Machine related.
[Frame] - Steam Frame related.
[Discussion] - General discussion.
[Help] - A request for help or support.
[News] - News about the deck.
[PSA] - Sharing important information.
[Game] - News / info about a game on the deck.
[Update] - An update to a previous post.
[Meta] - Discussion about this community.
If your post is only relevant to one hardware device (Deck/Machine/Frame/etc) please specify which one as part of the title or by using a device flair.
These are not enforced, but they are encouraged.
Rules:
- Follow the rules of Sopuli
- Posts must be related to Steam Hardware or Steam OS in an obvious way.
- No piracy, there are other communities for that.
- Discussion of emulators are allowed, but no discussion on how to illegally acquire ROMs.
- This is a place of civil discussion, no trolling.
- Have fun.
Try Vulkan in Koboldcpp https://github.com/LostRuins/koboldcpp/releases/tag/v1.81.1
I haven't heard of anyone trying it on deck but I don't see why it wouldn't work.
I probably can't help, I have ROCM acceleration working on my desktop linux PC for image gen and LLMs, but I don't think I had to install the massive rocm files for it to work.
What specifically are you trying to run using ROCM? If I remember right most of the stuff I tried just needed pytorch, which I would install in a python venv with pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.3 That's still pretty heft though, my current venv is 16GB in size, but may include some duplicate/old files from repeated upgrades without cleaning.
I'm mostly trying to run text-generation-webui for fun; what did you install to run image generation if you never installed the rocm binaries? Hopefully i should be able to replicate it in a container
Pytorch includes the necessary ROCm files for image gen as far as I know. My computer does have an /opt/ROCm folder but it's only a few megabytes in size.
Unfortunately I don't see any mention of pytorch in your linked program, and it does mention a proper ROCm install as a requirement there.
If that's the case i'm going to try to run it without installing rocm, since i saw a pytorch rocm package being downloaded by pip. I'm gonna need to reinstall though.
There are different pytorch install commands for cuda/ROCm. Usually the key to getting ROCm to work with AI GitHub projects is to edit a config file/launch option and replace the pytorch command with a ROCm one