this post was submitted on 20 May 2026
129 points (99.2% liked)

TechTakes

2577 readers
179 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] smeenz@lemmy.nz 4 points 1 day ago (2 children)

If sites start blocking googlebot en masse, then googlebot will just start ignoring robots.txt

[–] flamingos@feddit.uk 1 points 3 hours ago

Then you can just block the user agent in nginx or whatever you use, like all the other AI scrapers who ignore robots.txt (*cough* Amazon)

[–] HK65@sopuli.xyz 3 points 1 day ago

Can they just put an EULA on the site and then sue Google for unauthorized access?

Not in the US of course, but in the EU or something