Selfhosted

60934 readers

563 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

533

Using rsync for backups, because it's not shiny and new (tinkerbetter.tube)

submitted 9 months ago by mesamunefire@piefed.social to c/selfhosted@lemmy.world

137 comments fedilink hide all child comments

You might not even like rsync. Yeah it's old. Yeah it's slow. But if you're working with Linux you're going to need to know it.

In this video I walk through my favorite everyday flags for rsync.

Support the channel:
https://patreon.com/VeronicaExplains
https://ko-fi.com/VeronicaExplains
https://thestopbits.bandcamp.com/

Here's a companion blog post, where I cover a bit more detail: https://vkc.sh/everyday-rsync

Also, @BreadOnPenguins made an awesome rsync video and you should check it out: https://www.youtube.com/watch?v=eifQI5uD6VQ

Lastly, I left out all of the ssh setup stuff because I made a video about that and the blog post goes into a smidge more detail. If you want to see a video covering the basics of using SSH, I made one a few years ago and it's still pretty good: https://www.youtube.com/watch?v=3FKsdbjzBcc

Chapters:
1:18 Invoking rsync
4:05 The --delete flag for rsync
5:30 Compression flag: -z
6:02 Using tmux and rsync together
6:30 but Veronica... why not use (insert shiny object here)

you are viewing a single comment's thread
view the rest of the comments

[–] tal@olio.cafe 25 points 9 months ago* (last edited 9 months ago) (3 children)

I would generally argue that rsync is not a backup solution.

Yeah, if you want to use rsync specifically for backups, you're probably better-off using something like rdiff-backup, which makes use of rsync to generate backups and store them efficiently, and drive it from something like backupninja, which will run the task periodically and notify you if it fails.

rsync: one-way synchronization

unison: bidirectional synchronization

git: synchronization of text files with good interactive merging.

rdiff-backup: rsync-based backups. I used to use this and moved to restic, as the backupninja target for rdiff-backup has kind of fallen into disrepair.

That doesn't mean "don't use rsync". I mean, rsync's a fine tool. It's just...not really a backup program on its own.

[–] koala@programming.dev 1 points 9 months ago (1 children)

Beware rdiff-backup. It certainly does turn rsync (not a backup program) into a backup program.

However, I used rdiff-backup in the past and it can be a bit problematic. If I remember correctly, every "snapshot" you keep in rdiff-backup uses as many inodes as the thing you are backing up. (Because every "file" in the snapshot is either a file or a hard link to an identical version of that file in another snapshot.) So this can be a problem if you store many snapshots of many files.

But it does make rsync a backup solution; a snapshot or a redundant copy is very useful, but it's not a backup.

(OTOH, rsync is still wonderful for large transfers.)

[–] tal@olio.cafe 3 points 9 months ago (1 children)

Because every “file” in the snapshot is either a file or a hard link to an identical version of that file in another snapshot.) So this can be a problem if you store many snapshots of many files.

I think that you may be thinking of rsnapshot rather than rdiff-backup which has that behavior; both use rsync.

But I'm not sure why you'd be concerned about this behavior.

Are you worried about inode exhaustion on the destination filesystem?

[–] koala@programming.dev 3 points 9 months ago

Huh, I think you're right.

Before discovering ZFS, my previous backup solution was rdiff-backup. I have memories of it being problematic for me, but I may be wrong in my remembering of why it caused problems.

[–] neidu3@sh.itjust.works 1 points 9 months ago* (last edited 9 months ago)

+1 for rdiff-backup. Been using it for 20 years or so, and I love it.