Does the data change a lot? Does it need to be a block-based backup (e.g. bootable)? Otherwise, you could go with rsync or restic or borg to only refresh your backup copy with the changed files. This should be far quicker than taking a complete backup of the whole SSD.
this post was submitted on 16 Jan 2025
43 points (95.7% liked)
Selfhosted
60644 readers
589 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 3 years ago
MODERATORS
Thank you for the insight. Changes are incremental I suppose. You are correct that it’s more efficient. But I kind of want to back up the whole disk since I can keep a bootable drive with it right?
dd if=/dev/sda0 conv=sync,noerror bs=128K status=progress | gzip -c file.gz
You can add an additional pipe in there if you need to ssh it to another machine if you don't have room on the original.
The added info from pv is also nice ^^
If zstd is available, it is a lot more efficient and performant over gzip.
True. I've done that command for so long that I've kinda gotten gzip hardwired into my fingers.
I did a thing like this but with a ios command that wrote the disk to image and piped it to ssh but then piped it back to a waiting drive. It was great as you could pull the disk and boot right off it. Do you know if that can be done with dd?
I'd probably dd it straight on to the drive, but I'm sure you could get it to go to New Orleans and play the Macarana before it came back if you used enough pipes.
btrfs or zfs send/receive. Harder to do if already established, but by far the most elegant, especially with atomic snapshots to allow versioning without duplicate data.
You can use Rescuezilla which is basically a GUI fir Clonezilla, but easier to use 😋
Thanks guys. I went with Rescuezilla in the end. So far so good.
This is something I should consider!
My method requires that the drives be plugged in at all times, but it's completely automatic.
I use rsync from a central 'backups' container that pulls folders from other containers and machines. These are organized in
/BACKUPS/(machine/container)_hostname/...
The /BACKUPS/ folder is then pushed to an offsite container I have sitting at a friends place across town.
For example, I backup my home folder on my desktop which looks like this on the backup container
/BACKUPS/Machine_Apollo/home/dork/
This setup is not impervious to bitflips a far as I'm aware (it has never happened). If a bit flip happens upstream, it will be pushed to backups and become irrecoverable.
I see. This is more of a file system backup right? Do you recommend it over full disk backup for any reason? I can think of saving space.
I recommend it over a full disk backup because I can automate it. I can't automate full disk backups as I can't run dd reliably from a system that is itself already running.
It's mostly just to ensure that I have config files and other stuff I've spent years building be available in the case of a total collapse so I don't have to rebuilt from scratch. In the case of containers, those have snapshots. Anytime I'm working on one, I drop a snapshot first so I can revert if it breaks. That's essentially a full disk backup but it's exclusive to containers.
edit: if your goal is to minimize downtime in case of disk failure, you could just use RAID
I can’t automate full disk backups as I can’t run dd reliably from a system that is itself already running.
Can't you do a snapshot like VSS does on windows and back that up on a running system? I assume with a filesystem that supports snapshots that would be possible.
I'm sure there's ways to do it, but I can't do it and it's not something I'm keen to learn given that I've already kind of solved the problem :p
I’m on a similar boat except I might have less time resource available in the future cause I’m getting a job.
Hopefully I could automate full disk backup because if something like Immich breaks, I can just load up from the backup drive. My family also use the services so… I think it’s great you brought up RAID but I believe when Immich or any software mess things up it’s not recoverable right?
I think it’s great you brought up RAID but I believe when Immich or any software mess things up it’s not recoverable right?
RAID is not a backup, no. It's redundancy. It'll keep your service up and running in the case of a disk failure and allow you to swap in a new disk with no data loss. I don't know how Immich works but I would put it in a container and drop a snapshot anytime I were to update it so if it breaks I can just revert.
I use dd.
Based. Yes. It is an option.
Borg for files. Proxmox snapshots for the VMs.
Veeam endpoint free version is nice because it doesn't require a reboot.
I personally use Borg to do automatic backups.
Do you want to use desktop app or systemd timer?
Oh I've been using Acronis for this purpose for a while, nice to know foss tools exist that accomplish the same thing, I'll probably use this next time.
HD Clone X cloner Or MSP360 (Cloud Berry) Standalone Backup
Both cost money but not a lot and are very reliable.