this post was submitted on 19 May 2025
61 points (100.0% liked)
LocalLLaMA
2978 readers
91 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
~~Nice.~~ I always first look at the memory bandwidth if it's about AI. ~~And seems with a 224bit bus, they've done a better job than previous cards you'd find in that price segment.~~
Where do you see that? I thought the die was 192-bit at most.
There's is a specification for 224 GB/S.
Thank you very much for the correct information. I googled it and took the number from some random pc news page. Either they got it wrong or I might need new glasses. Nonetheless, 128 or 192-bit is what Intel has on their website. I wish they'd do more for cards with 16GB of VRAM or more. I think two hundred and something GB/s is about what Nvidia, AMD and everyone already did in their previous generation of graphics cards.
Prices for AMD/Nvidia (except maybe a used AMD 7900 XTX) are so awful that this is still a good deal, no matter how much bandwidth it has. For pure text LLM usage, capacity is king.
Intel's hands are tied buy what silicon they have available, unfortunately.
Hmm. I could buy a (new) Radeon 7600 XT right now for around 330€... that should be only slightly more than $300 plus VAT, and that also has 16GB of VRAM and a similar (slightly faster?) memory interface?
The better choice depends on which software stack you intend to wrangle with, how long you intend to keep the cards, and your usage patterns, but the B580 is the sorta newer silicon.
Exllamav3 is the shit these days (as you can fully offload 32Bs in 16GB with very little loss), and it's theoretically getting AMD support way before Intel (unless Intel changes that).
...Also, 2x 3060s may be a better option, depending on price.
It's difficult to make that decision. I mainly tinker with generative AI. Try a chatbot or agent or try creative writing with it, mess with FramePack or LTX-video, or explore some text to speech or whatever I find interesting when I got some time to spare.
Obviously I wouldn't spend a lot of money just to mess around. So yeah. I currently just rent cloud GPUs by the hour and I'm fine with that. Once we get a very affordable and nice AI card with lots if fast VRAM, I think I'm going to buy one. But I'm really not sure if this one or any previous generation Radeon is what I'm looking for. Or me spending quite some time on ebay to find some old 30x0. And as a Linux user I'm not really fond of the Nvidia drivers, so that also doesn't make it easier.
Oh yeah, you will run into a ton of pain sampling random projects on AMD/Intel. Most "experiments" only work out of the box on Nvidia. Some can be fixed, some can't.
A used 3090 is like gold if you can find one, yeah.
And yes, I sympathize with Nvidia being a pain on linux... though it's not so bad if you just output from your IGP or another card.
And yes, stuff rented from vast.ai or whatever is cheap. So are APIs. TBH thats probably the way to go if budget is a big concern, and a 24GB B60 is out of the cards.