this post was submitted on 08 Nov 2024
325 points (97.9% liked)

Selfhosted

60253 readers
561 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
 

I saw this post and I was curious what was out there.

https://neuromatch.social/@jonny/113444325077647843

Id like to put my lab servers to work archiving US federal data thats likely to get pulled - climate and biomed data seems mostly likely. The most obvious strategy to me seems like setting up mirror torrents on academictorrents. Anyone compiling a list of at-risk data yet?

you are viewing a single comment's thread
view the rest of the comments
[–] otter@lemmy.ca 74 points 2 years ago (6 children)

One option that I've heard of in the past

https://archivebox.io/

ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline.

[–] tomtomtom@lemmy.world 9 points 2 years ago (1 children)

I am using archivebox, it is pretty straight-forward to self-host and use.

However, it is very difficult to archive most news sites with it and many other sites as well. Most cookie etc pop ups on a site will render the archived page unusable and often archiving won’t work at all because some bot protection (Cloudflare etc.) will kick-in when archivebox tries to access a site.

If anyone else has more success using it, please let me know if I am doing something wrong…

[–] danielquinn@lemmy.ca 1 points 2 years ago

Monolith has the same problem here. I think the best resolution might be some sort of browser-plugin based solution where you could say "archive this" and have it push the result somewhere.

I wonder if I could combine a dumb plugin with Monolith to do that... A weekend project perhaps.

load more comments (4 replies)