this post was submitted on 28 Aug 2026
730 points (98.7% liked)

Technology

87624 readers
3890 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
 

For her safety, Doe has opted to receive alerts from the US Department of Justice Victim Notification System any time she may be a victim in a new criminal investigation. Although she has received countless alerts, she was shocked when the CCCP notified her that it had identified AI-generated CSAM on xAI that depicted her. This re-traumatized Doe, whose complaint alleged that messages were found on online forums “between offenders chatting about creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”

Now, Doe fears that xAI has not only made it easier to make more violative images of the most distressing time in her life, but also that xAI allegedly has stored the images that Grok generates and uses those outputs to further train Grok. Because of this, she believes that Grok has been trained on both the initial set of images that have haunted her for more than 20 years and the more recent AI-generated ones.

This is the first case to accuse xAI of training on CSAM, and the complaint does not go into great detail on that claim. Previously, Ars reported on a controversial dataset that was later scrubbed after researchers found CSAM in the training data, but there’s no indication xAI trained on that data. In a press release from lawyers representing Doe, it explained that Doe’s images were included in a CSAM Hash List maintained by NCMEC, and “that same material” allegedly “was part of the dataset xAI used to build Grok’s image and video generating capabilities.” The complaint similarly only alleged that “CSAM depicting Plaintiff with its longstanding well-known hash values has been used as a part of the dataset used by xAI.”

top 50 comments
sorted by: hot top controversial new old
[–] Treczoks@lemmy.world 2 points 55 minutes ago

If Doe is actually recognizable, then chances are high that actual pictures of her or him were used for training the AI. Wouldn't surprise me. They scrape the internet for everything they can get their digital hands on, regardless of copyright or criminal law. They are bound to find illegal stuff on that track.

The next thing is that there are probably confidential data in the training sets, either exposed by neglient users, or by hackers that breached sites and blackmailed them.

And of course all the copyrighted material they used without permission.

If AI companies would really get sued on those three illegal sources, they could probably close their doors.

[–] Mulligrubs@lemmy.world 21 points 4 hours ago* (last edited 4 hours ago) (2 children)

I don't believe ANY AI company using internet data to train AI has gone through all of the data to insure it's not absolute shite.

That seems pretty obvious when you realize that AI is stupid. Most people are stupid, AI is stupid. Tons of pedophiles, AI makes kiddy porn. Millions of racists, AI is racist.

Once I realized this, AI made sense. Checks out.

[–] skisnow@lemmy.ca 8 points 2 hours ago (1 children)

I'm morbidly curious about the engineers Musk hires, because overwhelmingly the most intelligent engineers and scientists I know wouldn't dream of applying to work at one of his companies. Even the ones that don't care about the Nazi shit have still read the many reports about how badly he treats his staff.

[–] grepe@lemmy.world 1 points 1 hour ago

i think part of the answer to your curiosity are IT layoffs. job market is shit right now. even though overwhelming majority of people would not work under these conditions if they could choose freely, when they have a mortgage and a family to take care of and they've been out of work for long enough the moral concerns and dignity can quickly be overriden by more pressing needs...

[–] mechoman444@lemmy.world 2 points 2 hours ago* (last edited 2 hours ago)

What? That's not even remotely how that works.

A human being is also shaped by the information they've encountered throughout their life. You've encountered racism, stupidity, misinformation, and countless other forms of bad information. That doesn't mean you automatically become racist, stupid, or incapable of distinguishing good information from bad information.

AI works similarly in the sense that its training data contains an enormous amount of contradictory, inaccurate, biased, and outright terrible information. The mere presence of that information in the training data does not logically imply that the resulting model possesses those characteristics.

And AI isn't "dumb" or "smart" in the way you're describing. Those are human cognitive attributes. The relevant question is what a model can actually do, what patterns it has learned, and how reliably it performs a given task.

You don't like AI. Fine. But you clearly don't understand how it works, and this is an unfortunately constant problem on this platform. People make an assumption about AI, then present that assumption as though they've discovered some fundamental truth about the technology.

Misinformation is misinformation. It doesn't become correct because you think the conclusion is morally justified. You didn't figure out anything about AI here. You made an assumption and called it true.

All, I'm saying if you're going to critique something, at least know what you're critiquing or criticizing.

Also before anybody makes any more assumptions, what happen here is absolutely mind-blowingly horrific. Using csam to train anything is just terrible and exploitive. I'm very sorry for doe for having to continuously go through this.

Also also the people responsible for this should be charged with possession of CSAM and prosecuted in a court of law.

Disclaimer: Disagreeing with someone's argument, pointing out flaws in their reasoning, or interpreting their position differently does not automatically make it a straw man. For example, if I say, "I think we should reduce military spending," and you respond, "You think we should completely eliminate the military," you have created a straw man. You are arguing against a position I never actually took.

[–] QueenHawlSera@piefed.social 32 points 9 hours ago (2 children)

I knew Elon was a pedophile (he begged to be on the island) but to actually train your AI on CSAM...

Well he should be arrested for possession of child pornography and Grok should be shut down until all of its CSAM is erased from its databanks

[–] Holytimes@sh.itjust.works 6 points 2 hours ago (1 children)

Unfortunately that's not really how the actual data is stored. Functionally there is no CSAM at all in its "databanks". Once Info goes through training what comes out the other side is just a mass of goop.

It's like if you took an entire cow ran it though a meat grinder. Then demanded that you remove only the chuck from the resulting ground beef.

It's not physically possible.

You can demand it's retrained entirely from the ground up with vetted data. To produce a higher quality clean dataset. And I would agree that is what should be done.

But you can't unground the beef

[–] Treczoks@lemmy.world 1 points 53 minutes ago

Do you really believe they just throw all the training data away after use?

[–] muusemuuse@sh.itjust.works 17 points 9 hours ago (1 children)

on the one hand, it makes sense if your goal is to train AI to recognize child porn as a simple binary state (bool isCP). Social media sites used to have humans looking at that stuff moderating from afar and it really takes a horrendous toll on their employees.

On the other hand, Elon has repeatedly shown he refuses to censor child porn. They didn’t train it to stop making kiddie porn. They trained it to create more kiddie porn. And that’s why he’s rich. Elon won’t say no. He doesn’t care.

[–] Nollij@sopuli.xyz 9 points 8 hours ago (1 children)

The detail about hash values is really important. The FBI maintains a database of known CSAM. Presumably, hers is in that database, hence the hash values. While not everyone has access to that DB, Xitter/etc does. There is no ambiguity of anything on that list; there's also no need for any human to review. It's already been confirmed.

While I'm not sure there's any case law about it, I would be amazed if using that to train generative AI (except POSSIBLY as content to block) was treated as anything other than possession, distribution, and maybe even production of CSAM.

Proving it might be difficult without full discovery, and AI is infamous for the massive corpus of training data. However, AI doesn't always generate truly unique works. Go to any image generator and prompt for a video game plumber, and you'll see an unmistakable image of Mario. It's possible that they can find a prompt that generates results close enough to her images.

[–] Holytimes@sh.itjust.works 1 points 2 hours ago

Prompting AI for a video game plumber is most likely going to result in a Mario like entity just because it's the most common example.

That's a poor example of what your talking about.

You need someone niche and narrow that has a much smaller sample size. It would be more like asking for a generic description of one persons fursona with out naming it. And the model spitting out a almost 1:1 copy of a real preexisting art work on the fursona. Because the model only has that one picture to base things off of. Which is a real problem.

It's also the best way to tell if a model was trained on something specific. General prompts aren't going to get you anything beyond just the fact that yes. X thing is popular enough that everyone and their grandmother creates content on it.

[–] Waterpumpee@lemmus.org 149 points 15 hours ago (2 children)

This should make the entire model illegal. Train it on illegal stuff: delete the whole model.

[–] Voroxpete@sh.itjust.works 45 points 13 hours ago (10 children)

In all seriousness, there are some very interesting legal questions that will be raised if this case makes it that far.

The problem is that there's no existing law that would effect this on its own. To my knowledge, no country in the world has a law on the books specifically dealing with AI models trained on CSAM. So the question, under existing laws, would turn on whether the data stored within the model itself would constitute CSAM.

The problem, in no small part, is that we have serious gaps in our public consensus knowledge about how LLMs actually work.

There's a case that, AFAIK, is still being argued in Germany pushing the theory that LLMs actually do, in effect, store a copy of all their training data, just in a compressed form. This certainly seems to hold some water given both the tests they relied on, and the situation with this Jane Doe where the model produced images so alike to real images of her that they tripped hash detections.

The German case argues that this is analogous to the difference between an MP3 and a WAV, or a JPEG and a PNG. That sharing a lossy copy of a work is no less infringing just because it's imperfect.

If the underlying claim - that LLMs function as a form of lossy compression - can be substantiated then there would be a real argument that the model itself would constitute CSAM. Since there would be no realistic method that I'm aware of for removing the offending material from the model - and presumably SpaceX would have to somehow prove that they've done so - that would make the entire model contraband. They'd have to retrain on a clean dataset.

Of course I said "if the case makes it that far" at the top because I don't think it will. SpaceX will do anything and everything to avoid handing over meaningful discovery in this case, including, I suspect, outright destruction of evidence. If there is anything that actually proves that they used CSAM in the training data then they are so far beyond fucked that there's simply no downside to further illegality in pursuit of concealing their crimes. They have the world's wealthiest asshole in a position to throw literal billions at making this go away. I genuinely wouldn't be surprised if people turn up dead off the back of this if that's what it takes.

[–] DementedSociety@lemmy.world 2 points 4 hours ago

God help us all

[–] cantstopthesignal@sh.itjust.works 17 points 11 hours ago (1 children)

It's illegal to posses. Anyone training the models on that is criminally liable for possession as is the company. And it's a conspiracy to posses that was directed by someone, which is now RICO. Will they prosecuted? No.

load more comments (1 replies)
[–] scrubbles@poptalk.scrubbles.tech 19 points 13 hours ago (1 children)

including, I suspect, outright destruction of evidence

If it was trained on it, that means they are in possession of it, which that right there is straight to jail. I have a feeling they're scrubbing everything they can right now as we chat

[–] CorneliusTalmadge@lemmy.world 7 points 12 hours ago (1 children)

Was going to say basically say the same exact same thing. There is no law saying how AI is handled when using stollen materials or other “illegal” content.

But somehow we have all been brought to believe that somehow “new” technology isn’t subject to existing laws.

If x or any other company downloaded CSAM everyone in the company should be arrested.

[–] Voroxpete@sh.itjust.works 6 points 10 hours ago (4 children)

If x or any other company downloaded CSAM everyone in the company should be arrested.

I assume you cannot possibly mean that as written, right?

I'm absolutely for arresting anyone who was involved with this, or had knowledge that it was happening. But we're obviously not talking about going after Jane the intern here, right?

[–] scrubbles@poptalk.scrubbles.tech 4 points 10 hours ago

I won't say everyone. I've actually been at a company who was investigated (not for CSAM, but other things that happened). I had no idea it even happened, and luckily was not involved with any of it. So for me no, I wouldn't have wanted that. That being said 2 things, say I had been in the position. If our scraper was downloading it and it was my scraper, damn right I would have flagged it to legal, HR, and everyone I could have, along with writing some way to prevent it, and written everything down in a complete log (off company computer). If it wasn't stopped immediately I would either quit, whistleblowed, or happily talked with anyone raiding and making sure any of the decision makers were hauled off. I don't blame someone for being lowest level at a shit company, been there. (Although I will say, xAI, come on, no one is "stuck" there, but that doesn't mean that Dave the brand new intern out of college should be hauled off). Who I blame are the suits who were probably told it was happening and chose to ignore it, and any engineer I do blame if they knew about it and chose not to do one of the above.

As an engineer I've had my fair share of let's say... challenges that I've had to morally grapple with. Things I've been asked to do that may not be moral. However, there's a pretty wide chasm between "Implement this dark pattern so people won't unsubscribe" and "host this and don't tell anyone"

load more comments (3 replies)
[–] silasmariner@programming.dev 1 points 7 hours ago (1 children)

Either people end up dead and yet it's still somehow a nothing burger or we don't even make it that far

[–] Voroxpete@sh.itjust.works 3 points 7 hours ago

Yeah, the stakes are just too high for SpaceX to let this get to trial. Any amount of illegality becomes worth it when you consider the alternative.

The only other possibility I can see is that they pull a Bungie; "perform an internal investigation", find an intern to blame for everything, and throw a huge settlement at Jane Doe. But even that would require an absolutely insane cover up to pull off.

load more comments (6 replies)
[–] pelespirit@sh.itjust.works 17 points 14 hours ago

Yep, shut that shit down. Also, pay her a shit ton of money.

[–] MummifiedClient5000@feddit.dk 155 points 17 hours ago (3 children)

Must suck to be a rich pedophile and not even get an invite to Epstein island.

[–] random_character_a@lemmy.world 79 points 16 hours ago (1 children)

Elon was too creepy for Epstein

[–] Sharkticon@lemmy.zip 83 points 16 hours ago (1 children)

You know I know you're joking, but I'm concerned people believe this. Because Elon did hang out with Epstein. There was one email where he was ignored because he was being too thirsty. But that's not the whole story. He loved the island so much he wanted to go back too hard.

[–] story@lemmy.zip 2 points 4 hours ago

thanks for the truth, truth guy :)

load more comments (1 replies)
[–] ShutUpWesley@piefed.zip 28 points 13 hours ago (2 children)

All billionaires are pedophiles

[–] story@lemmy.zip 2 points 6 hours ago* (last edited 6 hours ago) (2 children)

if we could expand this conversation, i think that to a degree the reverse is true.

i think the neurotypes (we might call them paraphilias actually but you'd be surprised how many people can be reduced to a paraphilia) that connect sex with power imbalance (age, wealth, intelligence, strength, or just legal context like contracts and political relationships) also tend to hoard (valuable things, money, information, connections, data, favors) and these tendencies strengthen each other in a stronger way than most people understand

if you're interested in what I'm saying, look up "impunity architecture", especially if you're Marxist because i think a lot of people think this is inherent to capitalism and it isn't at all.

[–] magnue@lemmy.world 1 points 3 hours ago (1 children)

My brain switched off slightly at the word neurotypes, then completely at the word paraphilias, FYI.

[–] story@lemmy.zip 1 points 1 hour ago

i wish for a world where we can communicate with experiences instead of words, we would understand each other so much better.

[–] ShutUpWesley@piefed.zip 1 points 5 hours ago (2 children)

I've always said that the biggest difference between a billionaire on a yatch and a homeless man with a cart of garbage is luck.

[–] uriel238@lemmy.blahaj.zone 1 points 3 hours ago

Billionaires usually start with a specific luck package that includes generational wealth and family connections. Then, a bunch of opportunity being at the right place at the right time.

If you don't have one of the first two then the opportunity has to do a whole lot more heavy lifting. Also, if you have enough generational wealth, then you can fail upwards, as did George W. Bush and Donald J. Trump.

Also plenty of people have the starting gen-wealth/family-connections package but not the luck of opportunity. And they will commonly end up with a solid career, but doesn't move them towards ultra-wealthy status. So millions or even tens of millions rather than hundreds of millions or billions.

And then there's the rare dude like Tom Anderson who made MySpace, sold it for about $500 million and has since spent the rest of his life traveling and going on fun adventures. But he would have been a musician or a philosophy professor if he didn't luck into making MySpace.

[–] story@lemmy.zip 1 points 4 hours ago

Tom Sexton likes to say that rich people are poor people with money

[–] schmups@lemmy.zip 1 points 6 hours ago (1 children)

Backward comments like this are why the conversation doesn't move.

[–] ShutUpWesley@piefed.zip 2 points 5 hours ago (1 children)

Can you explain why you think it's backwards? It may be hyperbolic and facetious, but I think recent history has shown a statistically significant correlation between the ultra-wealthy and pedophilia.

[–] BeliefsOnTrain@lemmy.world 1 points 2 hours ago (1 children)

Correlation doesn't imply causation does it?

I think that's another generalization based on indeed totally true recent news. But he is right that this is not at all helping the reasonning, hence "backward"

[–] ShutUpWesley@piefed.zip 1 points 1 hour ago

I don't think I was suggesting that being a billionaire causes you to be pedophile, or the other way around.

Do all statements that do not forward some sort of effort to reason through things count as backwards?

[–] truthfultemporarily@feddit.org 64 points 16 hours ago (1 children)
load more comments (1 replies)
[–] Blaster_M@lemmy.world 39 points 16 hours ago (3 children)

amazing... I know of smaller models that specifically avoided that sort of thing because even negative training could go wrong so easily... they could have avoided that thing entirely, but nope...

I could understand if they were training a safety classifier (a model that learns what danger stuff is so it can recognize it when it sees it) but I doubt this is what they were doing here. That and handling such a radioactive dataset makes handing the demon core seem safe.

[–] uriel238@lemmy.blahaj.zone 1 points 3 hours ago

I thought that Google used a similar dataset, possibly borrowing images from the NCVIP in order to create an analytic rule-set for which to omit images from Google Image Search. (A larger, similar rule-set is used to omit legal pornography when safe-search is on.)

Mind you, this was before the hyperscale AI era, when LLMs were things like SIRI and Google Now. And Google search still focused on websearch hits and not AI summaries.

Fun story: This was an area of study of mine during the early 2010s, since every image search engine would filter porn hits whether or not you had safe-search on or off. If it was turned off, porn would be filtered to the end of the list unless the engine decided you were intentionally looking for porn in which case the porn hits would be shown at the top of the list. There was no way to get results that ignored the ID-as-porn status of the images. I would enter ambiguously risqué terms to see how explicit I needed to be before the engine decided I was looking for porn.

[–] Gullible@sh.itjust.works 30 points 16 hours ago (2 children)

"You don't understand! I need my 400 terabytes of child pornography to keep the kids safe!"

load more comments (2 replies)
load more comments (1 replies)
load more comments
view more: next ›