this post was submitted on 14 Dec 2025
53 points (94.9% liked)

Selfhosted

59923 readers
808 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam.

  3. Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.

  4. Don't duplicate the full text of your blog or git here. Just post the link for folks to click.

  5. Submission headline should match the article title.

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
 

cross-posted from: https://discuss.online/post/32165111

I realize my options are limited, but what about any robots.txt style steps? Thanks for any suggestions.

you are viewing a single comment's thread
view the rest of the comments
[–] Nephalis@discuss.tchncs.de 2 points 6 months ago (1 children)

Isn't fail2ban a possibility too? I created a filter for chatgpt and some others, and it feels like its working. My radicale server is my only free acessable service but it comes with a small webgui and so the bots showed up. I have no clue if the bot gets a fraction of your site each time it shows up, but seemingly the ban happens within 300ms when I remember correct. So it wouldn't be that much of information...

When setting the retry to 1 it will ban at the first sight.

[–] JustTesting@lemmy.hogru.ch 2 points 6 months ago (1 children)

A big issue is that this works for bots that announce themselves as such, but there's lots that pretend to be regular users, with fake user agents and ips selected from a random pool with each ip only sending like 1-3 request/day, but overall many thousands of requests. In my experience a lot of them are from huawei and tencent cloud/ASN

[–] Nephalis@discuss.tchncs.de 2 points 6 months ago

Yes, if that is true (and I am not that suprised about it) it is nearly impossible to block them this way.