The way the law is being enforced now, this should be an entirely legitimate argument. A snowball's chance in hell though that it holds up without a legal team like OpenAI has.

That's all it should do. We're nowhere near an AI that could be an arbiter of truth. Hell, most AI couldn't even be trusted to parrot the literature accurately.

I would find this extremely useful as a tool to help me find sources that I then review myself - similar to how I use Wikipedia. But the danger is in people trying to use it for more.

Chat bots are a starting place. I find them useful for rubber ducking.

It seems like a good way to kick off a literature review.

I can't tell the difference between those and OP's.

Heads are rolling one way or the other no matter how many quarters you ch...err...uh...flip off.

We know that training LLMs on LLM-generated text leads to an absolute collapse in quality.

This is often repeated, and true. But needs to be qualified.

Modern LLMs use tons and tons of “augmented” data, which is code for LLM generated or massaged data. Some is even generated during training, and judged; papers on that are what made Deepseek famous.

Training on LLM trash will, of course, yield greater trash, and obviously good text has to come from something real. But that’s because slop is slop. And there are issues with “deep frying” LLMs, yes, but simply training on LLM on LLM output does not necessarily reduce quality. It often helps, significantly.

And we also know that AI has been showing up in papers so if they haven’t, then this will be quite unreliable.

Now this is a problem.

TBH LLMs would be pretty good at flagging papers for humans to check, similar to what Wikipedia is already doing. But yeah, if you just feed a prompt bad papers, LLMs just assume the context is true, generally, and that’s a tremendous problem.

You have to go into each article and check the key points, trust me.

It is a god-tier liar.

Yeah, it’s just a model with a semantic database it can query (RAG)

AI doesnt understand truth, it averages on data points. It cannot tell the "truth". it can be right sometimes based on frequency of mentioned words and related ones.

Of course not.

Yeah they added “Don’t hallucinate” to the prompt.

Have they solved the huge unsolved problem no one else has solved

yeah, no.

And without hallucinations ???

Likely not

It probably uses Retrieval Augmented Generation, which can still hallucinate, but usually does a better job for niche questions and it can even provide a source sometimes depending on how you set it up

LOL, of course not.

Speaking of hallucinations, I think the best way to see them is to go to Google Gemini (Reddit is selling them Reddit posts) and start a conversation about Reddit account you have and act as you don't know anything. It usually starts good but as it progresses you can see how it is making shit up. The more you ask the more insane it gets.

And this is supposedly having all the comments at its disposal.

I also tried Lemmy as I'm sure they are also indexing it. It is telling me that I'm actually admin who created Lemmy.dbzer0.com

Obviously not, because that's not possible.

perhaps you should unblock specific domains in that case

but i’d also suggest a blocker that uses auto-updated lists rather than whole gTLDs; they’re likely to catch more and deny fewer false positives