Ask Lemmy

It's proportional to the amount of rude AI crawling, from what I've heard.

The industry has put every reasonably accessible piece of human writing in their datasets, and are scrounging for the remaining little bits, now. Old, obscure, obsolete books nobody ever reads are apparently a hot item. Your latest blog post is probably worth some fraction of a penny.

I was running a mediawiki instance for a while, and despite the very low human traffic and arguably useless nature of the site to anyone but me and a few online friends I was getting hammered by Amazon. I don't think it got to the point of a denial of service but the traffic was excessive.

This was a problem before AI but has only gotten worse now that AI companies are scraping sites for training data.

CAPTCHAs and the like are often inaccessible to screen reader users, so the proliferation of CAPTCHAs has affected me greatly.

It is what it is I suppose.

Captchas suck. Don't trust reCaptcha, and avoid Cloudflare captchas.

That said, Techaro's Anubis is a good thing.

It is that or shutting down your site.

Those fucking bots just keep coming, causing traffic no small or medium site can cope with. So yeah, deal with it.

Im ok with them as long as theyre not demanding input by clicking everything that is a fire post so they can train their ai for free

Seems like a big switch in the last 3-6 months? Even google is making me do stupid captcha when I occasionally use it.

In the last 24 hours my small personal website got 8 legitimate visitors and 74,000 requests from bots, most of which were either malicious or aggressive ai crawlers

Just the "click and hold" ones that take forever and only work 20% of the time. I need a damn good reason to go there, else I say "fuck it" and move on.

Since I use a VPN they are everywhere. Small price to pay.

They're frustrating mostly because they're likely not working.

Oh sure, they're preventing a lot of low effort bots and scrapping but so many more get through.

My Lemmy bot-check comes with a picture of a sexy cat girl. It's brief, but it's there.

lemmy.zip only has these if you're not logged in

Using a VPN? They trigger like whoA

yes, including useless captchas.. even more so as i know they're trivially bypassed/workarounded on sophisticated bots

i understand it does block stupid crawlers but still.

Unless you want this place to wind up like youtube and reddit with zionist AI bots skewing the entire website in favor of israel, no.

My biggest problem with this kinda stuff is with Anubis and like one site using cloudflare.

With my current setup, like what feels like 99% of alternative r€ddit alternate frontend hosted sites don't let me in. So I am stuck using my redirect extension to keep trying to connect to a different instance from the list and it usually takes a while.

And then, as someone who like emulation, I gotta give a huge screw you to how ghe emulation gametech wiki thing completely blocks me from connecting to their site with a VPN active. Like, I just wanna see what's new in the emulator realm in terms of whether or not they still suggest certain emulators and their cloudflare settings basically shit on me for wanting to provately browse their website. Fuck me for not wanting my ISP looking at everything I do!

They should just use Anubis. At least we then see cute anime characters while the bot check runs.

I've just considered it a casualty of the privacy wars.

I don't know if they're deployed as a "weapon of annoyance" or anything, but I do notice that when I'm using a) a degoogled browser, b) an ad-blocking dns, c) mask my location, and/or d) block my IP. Suddenly (almost) every website makes me wait by throwing that at me.

I think it's lowkey trying to annoy me to the point where I give up on having those privacy measures, but they underestimate my hatred of corporate fuckery.

Bots are genuinely a problem.

Nah, fuck them bots.

I'm going to go ballistic if I have to pick out all the pictures of fucking crosswalks one more time.

Cloudflare tracks TF outta you. Not antibot checkoff, but Cloudflare def