overview for rkd

I Took Bernie Into Deep Trump Country. Can He Win Them Over? in c/unions@sh.itjust.works

[–] rkd@sh.itjust.works 3 points 2 months ago

I can read minds and they're thinking "we better get some money around here, otherwise we're still blaming the immigrants".

What makes open-source apps great? in c/CoMaps@sopuli.xyz

[–] rkd@sh.itjust.works 11 points 3 months ago

If they're not great, it's your fault /thread 😅

So image generation is where it's at? in c/localllama@sh.itjust.works

[–] rkd@sh.itjust.works 1 points 3 months ago

I believe right now it's also valid to ditch NVIDIA given a certain budget. Let's see what can be done with large unified memory and maybe things will be different by the end of the year.

monday is coming up in c/memes@lemmy.world

[–] rkd@sh.itjust.works 2 points 3 months ago

can't have both

The Trouble With Trump’s Deal With Nvidia And AMD: It’s An Export Tax in c/economy@lemmy.world

[–] rkd@sh.itjust.works 0 points 3 months ago (1 children)

chat is this socialism

European leaders including Starmer to join Zelenskyy in Washington for meeting with Trump in c/ukraine@sopuli.xyz

[–] rkd@sh.itjust.works 6 points 3 months ago

no more fokin ambushes

Three killed, eight injured in shooting in crowded New York club in c/news@endlesstalk.org

[–] rkd@sh.itjust.works 1 points 3 months ago

Trump has entered the chat

Trump made direct financial demands during call with Swiss president in c/news@endlesstalk.org

[–] rkd@sh.itjust.works 33 points 3 months ago (1 children)

His whole existence is a financial demand. I believe Bloomberg calls this "a transactional period". Put it plainly, y'all elected a corrupt president.

HP Z2 Mini G1a Review: Running GPT-OSS 120B Without a Discrete GPU in c/localllama@sh.itjust.works

[–] rkd@sh.itjust.works 1 points 3 months ago* (last edited 3 months ago)

For some weird reason, in my country it's easier to order a Beelink or a Framework than an HP. They will sell everything else, except what you want to buy.

GPT-OSS 20B and 120B Models on AMD Ryzen AI Processors in c/localllama@sh.itjust.works

[–] rkd@sh.itjust.works 1 points 3 months ago (1 children)

Remind me of what are the downsides of possibly getting a framework desktop for christmas.

So image generation is where it's at? in c/localllama@sh.itjust.works

[–] rkd@sh.itjust.works 1 points 3 months ago

That's a good point, but it seems that there are several ways to make models fit in smaller memory hardware. But there aren't many options to compensate for not having the ML data types that allows NVIDIA to be like 8x faster sometimes.

So image generation is where it's at? in c/localllama@sh.itjust.works

[–] rkd@sh.itjust.works 1 points 3 months ago

For image generation, you don't need that much memory. That's the trade-off, I believe. Get NVIDIA with 16GB VRAM to run Flux and have something like 96GB of RAM for GPT OSS 120b. Or you give up on fast image generation and just do AMD Max+ 395 like you said or Apple Silicon.

14

So image generation is where it's at? (sh.itjust.works)

submitted 3 months ago by rkd@sh.itjust.works to c/localllama@sh.itjust.works

15 comments fedilink

Total noob to this space, correct me if I'm wrong. I'm looking at getting new hardware for inference and I'm open to AMD, NVIDIA or even Apple Silicon.

It feels like consumer hardware comparatively gives you more value generating images than trying to run chatbots. Like, the models you can run at home are just dumb to talk to. But they can generate images of comparable quality to online services if you're willing to wait a bit longer.

Like, GPT OSS 120b, assuming you can spare 80GB of memory, is still not GPT 5. But Flux Shnell is still Flux Shnel, right? So if diffusion is the thing, NVIDIA wins right now.

Other options might even be better for other uses, but chatbots are comparatively hard to justify. Maybe for more specific cases like code completion with zero latency or building a voice assistant, I guess.

Am I too off the mark?