Self-hosting

3961 readers

46 users here now

Hosting your own services. Preferably at home and on low-power or shared hardware.

Also check out:

founded 3 years ago

MODERATORS

poVoq@slrpnk.net

sam_uk@slrpnk.net

Guarding My Git Forge Against AI Scrapers (vulpinecitrus.info)

submitted 1 month ago by tofu@lemmy.nocturnal.garden to c/selfhosting@slrpnk.net

2 comments fedilink hide all child comments

Didn't read the full post yet but the introduction on how much load these carpets create was interesting.

you are viewing a single comment's thread
view the rest of the comments

[–] keepthepace@slrpnk.net 2 points 1 month ago* (last edited 1 month ago)

Ads make the web human-hostile and such disgraceful behaviors by scrapping bots force to make it bot-hostile too.

I am sad and depressed. I went into AI to solve the problems of the world and I still think that the progresses made in machine learning are a huge step to to improve the world, but seeing what capitalism turned these tools into... sigh.

I don't even have the strength anymore to explain that "AI companies" and "AI"are not the same thing...

There should be ways to behave correctly. Robot.txt should be legally enforced, rate limitation should be respected and prosecuted. Sites with information they are willing to share with models should just provide a datadump and individual requests should be reserved for human usage.

But an internet where everyone is understanding of each other and business actors do not act like psychopaths does not exist.