This is a most excellent place for technology news and articles.
Bruh, this is literally like scientists scavenging metal from shipwrecks that predate the era of nuclear bomb testing, cause they need steel that isn't contaminated by nuclear fallout for certain measuring equipment π΅βπ«π΅βπ«
Great analogy for the shit we're currently saturating our digital world with
Low background bullshit training data.
A guy I used to know who would go to estate sales looking for blacksmithing equipment once told me the market for old metalworking stuff is disappearing because too much stuff is being bought by companies Γ©n masse for this reason.
I don't know if I believe that companies are buying small lot auctions and estate sales for old anvils, but he was sure convinced.
I like to joke I'm sitting on a pile of cash because my anvil isn't radioactive. (it's not a very good joke, but it carries some weight!)
According to Wikipedia, while a market for old, low-background steel was real phenomenon for a few decades, that's pretty much over.
World anthropogenic background radiation ... began to fall in 1963 ... and by 2008 it had decreased to only 0.005 mSv/yr above natural levels.[9] This has made special low-background steel no longer necessary for most radiation-sensitive uses, as new steel now has a low enough radioactive signature.
So you may consider your friend's estate-sale theory in light of that time-line.
but it carries some weight
Did you know about !dadjokes@lemmy.world?
If a big company needed a lot of something specific, they'd probably pay a pretty penny and let people going to small lot auctions accumulate a sizeable amount and then buy it from them.
It's not impossible to buy stuff from small lots at scale, as long as you're willing to pay a premium
It sounds plausible enough, considering we apparently have physicists out there using lead from roman archeological artifacts in their contraptions π https://www.scientificamerican.com/article/ancient-roman-lead-physics-archaeology-controversy/
Bruh, this is literally like scientists scavenging metal from shipwrecks that predate the era of nuclear bomb testing, cause they need steel that isnβt contaminated by nuclear fallout for certain measuring equipment
Except in this case, these scientists are working at a nuclear weapons lab. They need metal that hasn't been contaminated by nuclear bombs so that they can invent better nuclear bombs.
Fking nailed it.
The worst part is that they are destroying the books afterwards.
Reminds me of Judge Holden in Blood Meridian, who sketched ancient artifacts in his personal notebook and then destroyed the originals.
I'm waiting for the day when desperate LLM companies start paying people to post real human content, only for those people to just ask ChatGPT to do it.
"now you only posted 4 times today Jenny, do you just want to do the minimum?"
This is already happening.
They are paying people bottom market rates to generate unique written content and people are just having AI write it for them.
Destroying the books too :(
Not to defend the AI crowd, but ever go to a Goodwill Outlet store? They usually have at least 1 gaylord box (the giant box on a pallet watermelons come in) full of books. Not sure how often they cycle those out, but I'm guessing pretty regularly. Multiply that by every semi major city Goodwill has a presence in. Add in other thrift sores with a similar quantity of books. You're talking ALOT of books. SO MANY books are thrown out everyday. Including by libraries. I don't really have a problem if some of those get diverted for a bit to be scanned. Yes, it's sad to see books getting destroyed. But there's alot of stuff people just really don't want. And, if you REALLY want a villain, maybe you can blame copyright law? That's part of the reason AI companies destroy the books - it helps to make their scanning efforts more legal. Absent that, they could probably be convinced to spend just a little more for nondestructive scanning, assuming they could find a home for the books they scanned (the sellers of these things couldn't, so good luck).
If the AI companies were simply picking up books that were already going to a landfill, I don't think that would bother anybody. That isn't what we're talking about here, though. These books are coming from booksellers, so it's fair to guess that most of them are at least a notch or two above what you'd find in a Goodwill bargain bin.
You can only stave off model collapse so long.
Itβs only a matter of time before most of the data going in to LLMs came out of one too.
Like Iβm not even good at math or anything, much less computer science, but that seems obvious and inevitable to me.
Maybe someone smarter than me can chime in.
It's kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.
Damn AI ruined AI.
Hee hee.
Anna's Archive: ...uh, well okay, then.
Nice that they're actually buying them this time instead of just downloading torrents of book collections again.
I imagine they already burned through literally all of the torrentable books
That would be all the books.
Far from it. I think you vastly underestimate how many books have been published, and in what small quantities.
Estimates are around 15% of books is digitised, and less than a third of that is searchable. Iβm guessing a small fraction of that is torrentable (yes thatβs a word I just made up).
Modern book burning
Ah fuck... No wonder old maths texts are a nightmare to get hold of recently...
They first crashed RAM prices, the SSDs, and now books too?! Oh fuck π
Why? So their LLMβs can ingest the books and use them to churn out more AI slop?
They're doing everything they can to avoid it, but it's supposed to be tHe FuTuRe
