this post was submitted on 18 Dec 2025
176 points (98.4% liked)

Fuck AI

4903 readers
2423 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS
VerbFlow@lemmy.world
MrMcGasion@lemmy.world
TootSweet@lemmy.world
BigMikeInAustin@lemmy.world
cynar@lemmy.world
drmeanfeel@lemmy.world
pavnilschanda@lemmy.world
CriticalMedicine@lemmy.world
WonderfulWanderer@lemmy.world
Communist@lemmy.ml
eatCasserole@lemmy.world
SpaceNoodle@lemmy.world
NutWrench@lemmy.world
Soup@lemmy.cafe
iAvicenna@lemmy.world
Tinks@lemmy.world
wizblizz@lemmy.world
corus_kt@lemmy.world
Prandom_returns@lemm.ee
JimSamtanko@lemm.ee
TrickDacy@lemmy.world
TheFriar@lemm.ee
ArmokGoB@lemmy.dbzer0.com
HawlSera@lemm.ee
andrew_bidlaw@sh.itjust.works
MeDuViNoX@sh.itjust.works
33550336@lemmy.world
Nougat@fedia.io
Lost_My_Mind@lemmy.world
Sterile_Technique@lemmy.world
Quill7513@slrpnk.net
glowing_hans@sopuli.xyz
e8d79@discuss.tchncs.de
ThefuzzyFurryComrade@pawb.social
176
Nightshade - A new data poisoning tool lets artists fight back against generative AI (lemmy.world)
submitted 14 hours ago by ekZepp@lemmy.world to c/fuck_ai@lemmy.world
16 comments fedilink hide all child comments
 

cross-posted from: https://lemmy.world/post/7258145

The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models. Is intended as a way to fight back against AI companies that use artists’ work to train their models without the creator’s permission.

ARTICLE - Technology Review

ARTICLE - Mashable

ARTICLE - Gizmodo

The researchers tested the attack on Stable Diffusion’s latest models and on an AI model they trained themselves from scratch. When they fed Stable Diffusion just 50 poisoned images of dogs and then prompted it to create images of dogs itself, the output started looking weird—creatures with too many limbs and cartoonish faces. With 300 poisoned samples, an attacker can manipulate Stable Diffusion to generate images of dogs to look like cats.

top 16 comments
sorted by: hot top controversial new old
[–] ZDL@lazysoci.al 4 points 4 hours ago (1 children)

Isn't the easiest way to poison the degenerative AI pool to just feed it degenerative AI output?

[–] hildegarde@lemmy.blahaj.zone 3 points 1 hour ago

This tool is for artists to protect their own works from theft. This tool watermarks the art in a minor way that is difficult for humans to notice, but messes up current AI models that use it as training data.

Yes AI incest does degrade the models, but that strategy is ineffective at protecting the works of artists.

[–] dejected_warp_core@lemmy.world 8 points 6 hours ago

What's old is new again.

https://en.wikipedia.org/wiki/Fictitious_entry

Fictitious or fake entries are deliberately incorrect entries in reference works such as dictionaries, encyclopedias, maps, and directories, added by the editors as copyright traps to reveal subsequent plagiarism or copyright infringement. There are more specific terms for particular kinds of fictitious entry, such as Mountweazel, trap street, paper town, phantom settlement, and nihilartikel.[1]

Disregarding the obscene amount of automation at play, the underlying problem and its remedy remain somewhat the same. Put bad data into a very large dataset, such that it evades cursory scans, and the unaware plagiarists are eventually caught red-handed.

[–] Blackfeathr@lemmy.world 42 points 14 hours ago* (last edited 14 hours ago) (1 children)

Unfortunately, Nightshade has already been thwarted by LightShed with 99.8% accuracy:

https://cybernews.com/ai-news/glaze-nightshade-cant-stop-ai-scraping-art/

https://www.technologyreview.com/2025/07/10/1119937/tool-strips-away-anti-ai-protections-from-digital-art/

https://www.cam.ac.uk/research/news/ai-art-protection-tools-still-leave-creators-at-risk-researchers-say

[–] MartianSands@sh.itjust.works 20 points 13 hours ago (2 children)

That's the problem with all of these attempts. They treat these "poisons" as if they work on AI in general, when in fact they're very specifically created to target specific models.

Not only will they only work on some AIs, it's not terribly difficult to modify the AI enough that it needs a different poison

[–] UnspecificGravity@piefed.social 21 points 13 hours ago (1 children)

Just like with poisonous creatures in nature its not about just killing everything that tries to eat you its about making it easier to eat something else. Having to CONSTANTLY develop new strategies in order to train their models on artwork increases the cost to maintain this practice. Eventually it raises it high enough that the cost isn't worth the result.

[–] Voroxpete@sh.itjust.works 1 points 9 hours ago (2 children)

Also, I'd really like to know how much additional processing time is required to de-nightshade an image? And how much is required to detect nightshade, if that's even a different amount? Do you just have to de-nightshade every image to be safe?

Suppose the workload of de-nightshading is equal to the workload of training on that image. You've just doubled training costs. What if it's four times? Ten times?

That de-nightshading tool works in a lab, sure, but the real question is if it scales in a practical and cost effective way. Because for each individual artist the cost of applying nightshade is functionally nil, but the cost for detecting / removing it could be extremely high.

[–] ZDL@lazysoci.al 1 points 4 hours ago

Well degenerative AI in general doesn't scale in a practical and cost effective way, so ... I think the conclusion for de-nightshading is obvious?

[–] UnspecificGravity@piefed.social 2 points 9 hours ago

Plus they have to keep developing solutions to Nighshade 2 and Nightshade 2.1 and the Deathcap fork etc. etc. An enthusiastically developed open source project with a bunch of forks and versions is not an easy thing for a big lumbering corporation to keep up with. Especially a corporation that is actively trying to replace staff with AI coders.

[–] finitebanjo@lemmy.world 8 points 13 hours ago* (last edited 13 hours ago)

Idk, I'm not convinced. A lot of AI develpment gets treated like a black box algorithm. Overconstraining can lead to model collapse due to more convergent behavior than normal.

[–] ceenote@lemmy.world 19 points 14 hours ago (1 children)

I give it one month before Donald Trump signs a bill outlawing the poisoning of AI models with bipartisan support.

[–] finitebanjo@lemmy.world 5 points 13 hours ago (1 children)

"with bipartisan support" is where you've lost me.

[–] ceenote@lemmy.world 9 points 12 hours ago* (last edited 12 hours ago)

Plenty of politicians on both sides are in big tech's pockets. When it's a Democrat it's a bug, when it's a Republican it's a feature.

[–] Blaster_M@lemmy.world 3 points 13 hours ago (2 children)

It's trivial to defeat this, and at the levels you need to really make it work, your image looks terrible. Don't publicly share something if you don't want it to get in a dataset somehow.

[–] dejected_warp_core@lemmy.world 3 points 6 hours ago

Eh, it's not hard to hide things away in a website that only a bot (or a very determined human) will find. You don't have to poison all your images, just enough of them.

[–] SereneSadie@quokk.au 1 points 6 hours ago

Right, because artists are all about hoarding their work to themselves and not letting anyone get copies ever.

They'll have to DRM it by restricting access to being there in person only, no recording devices whatsoever.

Clearly thats the only logical solution left.