this post was submitted on 17 Jun 2025
81 points (100.0% liked)
Technology
39441 readers
277 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
That suggestion is exactly the same as what I started with when I said "IMO the ideal solution would be the one Wikimedia uses, which is to make the information available in an easily-downloadable archive file." It just cuts out the Aaron-Schwarts-style external middleman, so it's easier and more efficient to create the downloadable data.
They put the website up. Load balancing, rate limiting, and such go with the turf. It's their responsibility to make the site easy to use and hard to break. Putting up an archive of the content that the scrapers want is an easy and straightforward thing to do to accomplish this goal.
I think what's really going on here is that your concern isn't about ensuring that the site is up, and it's certainly not about ensuring that the data it's providing is readily available. It's that there are these specific companies you don't like and you just want to forbid them from accessing otherwise freely accessible data.
Yes. Which is why I'm suggesting providing an approach that doesn't require scraping the site.
Perhaps be more succinct? You're really flooding the zone here.
No, I'm staying focused.
"be more succinct"?
maybe have AI summarize it for you 🙄
Do you have any idea how deeply it undermines your argument when you just openly say, "You're writing oo much for me to read, please write less."
Don't respond if you don't have the common courtesy to read what the person person wrote.