this post was submitted on 13 Jan 2026
303 points (97.8% liked)

Technology

78661 readers
3289 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 

Website operators are being asked to feed LLM crawlers poisoned data by a project called Poison Fountain.

The project page links to URLs which provide a practically endless stream of poisoned training data. They have determined that this approach is very effective at ultimately sabotaging the quality and accuracy of AI which has been trained on it.

Small quantities of poisoned training data can significantly damage a language model.

The page also gives suggestions on how to put the provided resources to use.

top 50 comments
sorted by: hot top controversial new old
[–] vane@lemmy.world 12 points 9 hours ago

I have around 10-20GB github / gitlab mirror. I am constantly under attack from crawlers from top US technology corporations and LLM startups. Whenever I ban one IP range they switch to other - I don't know if those fuckers have tickets in their systems to do it manually or they just deploy this shit all over the planet. From what I observe during attacks that I mitigate the best way to poison them is to just create gitea instance with poisoned code repository and couple hundred revisions. It's because what they are most interested in is html representation of diff between two git revisions.

[–] Blackmist@feddit.uk 4 points 9 hours ago

With the amount of AI generated horseshit out there already, they've already pissed in the well.

[–] nutsack@lemmy.dbzer0.com -2 points 8 hours ago* (last edited 8 hours ago) (1 children)

I don't think this is a good idea. The pollution spreads. this would corrupt the collective knowledge of humanity a little faster than the AI already is doing.

[–] phoenixz@lemmy.ca 1 points 7 hours ago

Nah, AI will already do that automatically because any and all system loses something in inefficiencies. Same like if you put a theoretical 100 miles of gas worth in your tank that turns into 20 in practice because the combustion engine has an efficiency of 30ish%, you have air and tire resistance, etc etc.

AI has the same for information, and what comes out is always a certain fraction of the 100% that went in

Since poisoning the pool makes AI unreliable up to the point where it becomes useless, it has the potential to stop the AI madness. I'd be all for that.

[–] termaxima@slrpnk.net 46 points 1 day ago (1 children)

Been thinking about making one of these too, especially since I have a catchy name : asbestos

[–] ZILtoid1991@lemmy.world 4 points 17 hours ago

Me too, but with procedural image generation. Use some templates which are put together with CPU blitter (extremely fast and effective), add some random descriptive text, then done. Don't know how much my theory would work IRL.

[–] eru@mouse.chitanda.moe 7 points 1 day ago (2 children)

i would imagine companies would just filter it out

need some more clever way of hiding it or allow it to be self hosted so that it has various urls

[–] CileTheSane@lemmy.ca 5 points 8 hours ago

So it would be effective at preventing your site from being used as training data.

[–] GamingChairModel@lemmy.world 18 points 1 day ago

If I am reading this correctly, anyone who wants to use this service can just configure their HTTP server to act as the man in the middle of the request, so that the crawler sees your URL but is retrieving poison fountain content from the poison fountain service.

If so, that means the crawlers wouldn't be able to filter by URL because the actual handler that responds to the HTTP request doesn't ever see the canonical URL of the poison fountain.

In other words, the handler is "self hosted" at its own URL while the stream itself comes from the same URL that the crawler never sees.

[–] vacuumflower@lemmy.sdf.org 12 points 1 day ago (5 children)

If, suppose, I were optimistic over this technology, but pessimistic over its current stage of development, I'd expect this to be a cure. It's a problem they'll have to solve. A test they'll have to pass.

If somewhere inside those things someone makes a mechanism building a graph of syllogisms, no kind of poisoned input data will be able to hurt them.

So - this is a good thing, but when people say it's a rebellion, it's not.

You ascribe far too much to the internal workings than is reasonable.

[–] kadu@scribe.disroot.org 17 points 1 day ago (4 children)

Samsung and Anthropic published independently created data showing how little bad data it takes to effectively poison very large models. LLMs pretend to be complex, but they aren't, they'll not continue to improve at the initial rate we got used to seeing. Just ask OpenAI.

load more comments (4 replies)
[–] FlashMobOfOne@lemmy.world 22 points 1 day ago (2 children)

A test they’ll have to pass.

This makes me chuckle, as they invented euphemisms like 'hallucinations' because their LLM models can't do what they promise. Fabulous marketing, but clearly they didn't do enough testing.

[–] Bazoogle@lemmy.world 4 points 1 day ago (1 children)

as they invented euphemisms like ‘hallucinations’

Seems like a pretty accurate word to use, no? Could also use fabrication, concoction, phantom, or something else? I think "lie" and its synonyms are not accurate, since that requires intent. Since the LLM does not have intent, it cannot "lie".

[–] GamingChairModel@lemmy.world 10 points 1 day ago

That's why "bullshit," as defined by Harry Frankfurt, is so useful for describing LLMs.

A lie is a false statement that the speaker knows to be false. But bullshit is a statement made by a speaker who doesn't care if it's true or false.

load more comments (1 replies)
[–] Disillusionist@piefed.world 22 points 1 day ago

Not all problems may be cured immediately. Battles are rarely won with a single attack. A good thing is not the same as nothing.

[–] treesapx@lemmy.world 5 points 1 day ago (1 children)

"You're not opposing me. All you've done is create a problem that will stop me until I have it figured out." is the description of every struggle between opposing forces, so it's interesting that you disagree with that.

load more comments (1 replies)
load more comments
view more: next ›