lol nice BSD brag thrown in there
this post was submitted on 15 Dec 2025
722 points (98.5% liked)
Technology
77742 readers
3323 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
I don't know that it's wise to trust what anthropic says about their own product. AI boosters tend to have an "all news is good news" approach to hype generation.
Anthropic have recently been pushing out a number of headline grabbing negative/caution/warning stories. Like claiming that AI models blackmail people when threatened with shutdown. I'm skeptical.
load more comments
(1 replies)
found that with just 250 carefully-crafted poison pills, they could compromise the output of any size LLM
That is a very key point.
if you know what you are doing? Yes, you can destroy a model. In large part because so many people are using unlabeled training data.
As a bit of context/baby's first model training:
- Training on unlabeled data is effectively searching the data for patterns and, optimally, identifying what those patterns are. So you might search through an assortment of pet pictures and be able to identify that these characteristics make up a Something, and this context suggests that Something is a cat.
- Labeling data is where you go in ahead of time to actually say "Picture 7125166 is a cat". This is what used to be done with (this feels like it should be a racist term but might not be?) Mechanical Turks or even modern day captcha checks.
Just the former is very susceptible to this kind of attack because... you are effectively labeling the training data without the trainers knowing. And it can be very rapidly defeated, once people know about it, by... just labeling that specific topic. So if your Is Hotdog? app is flagging a bunch of dicks? You can go in and flag maybe 10 dicks and 10 hot dogs and ten bratwurst and you'll be good to go.
All of which gets back to: The "good" LLMs? Those are the ones companies are paying for to use for very specific use cases and training data is very heavily labeled as part of that.
For the cheap "build up word of mouth" LLMs? They don't give a fuck and they are invariably going to be poisoned by misinformation. Just like humanity is. Hey, what can't jet fuel melt again?
load more comments
(1 replies)
On that note, if you're an artist, make sure you take Nightshade or Glaze for a spin. Don't need access to the LLM if they're wantonly snarfing up poison.
the reason more people haven't adopted that is because they don't work.
I haven't seen any objective evidence that they don't work. I've seen anecdotal stories, but nothing in the way of actual proof.
You can't prove a negative, what you should look for is evidence that it works, without such evidence, there is no reason to believe it does.
Okay. I have that. Now what?
ETA: also, you can prove a negative, it's just often much harder. Since the person above said it doesn't work, the positive claim is theirs to justify. Whether it's hard or not is not my problem.
Okay. I have that. Now what?
Then you have your evidence, and your previous post is nonsensical.
That's not how evidence works. If the original person has evidence that the software doesn't work, then we need to look at both sets of evidence and adjust our view accordingly.
It could very well be that the software works 90% of the time, but there could exist some outlying examples where it doesn't. And if they have those examples, I want to know about them.
Last time I checked out Glaze, around the time it was announced, they refused to release any of their test data, and wouldn’t let people test images they had glazed. Idk why people wouldn’t find it super sus behavior, but either way it’s made moot by the fact that social media compresses images and ruins the glazing anyway, so it’s not really something people creating models worry about. When an artist shares their work, they’re nice enough to deglaze it for us.
Well I haven’t seen any objective evidence that god doesn’t exist, but that don’t mean I believe in her.
Okay. Same. I'm not asking you to believe Glaze/Nightshade works on my word alone. All I said was that artists should try it.
if that's true, why hasn't it worked so far then?
I used to think it wasn't viable to poison llms, but are you saying there's a chance? [a meme comes to mind]
You and me. We just need 248 more volunteers and we can save the world!
Isn't this applicable to all human societies as well though?
Well, I'm still glad offline LLM's exist. The models we download and store are way less popular then the mainstream, perpetually online ones.
Once I beef up my hardware (which will take a while seeing how crazy RAM prices are), I will basically forgo the need to ever use an online LLM ever again, because even now on my old hardware, I can handle 7 to 16B parameter models (quantized, of course).
So what websites should be targeted?