I finally got some online storage to backup my home server to, and I can finally sleep easy when my house inevitably goes up in flames. My initial push has been with rclone using the sync command. I plan on running a daily or weekly push.
I’ve gone with Hetzner and they have a variety of options on how to connect to their storage box as they call it. They have an option for automatic snapshotting of, as I understand it, the entire storage box that you can then recreate.
So my thoughts are around rclones sync and copy commands, and the storage snapshots.
Copy) If I use copy, it won’t clear up deleted files automatically that I actually want gone.
Sync) With sync I’m worried that say a drive failed (my set up is a bit wonky currently until I can build a proper RAID), it will do a sync, see an entire directory is gone, and go ahead and delete it.
Snapshots) Here I’m worried that any new data that’s been added in-between the snap shot and a server failure would be lost. And I don’t you can retrieve partial data from a snapshot.
What I have in mind is to use rclone’s copy command, but then somehow have a retention period on files that no longer exist on my server. So a deleted file might exist in my storage bucket for a month before being removed from there as well.
How do you guys manage this? Are there any ready made solutions, or do I need to whip up a script that goes through and looks at the last modified dates and removes the old ones?
Scripting it isn’t that tricky.
I’m old and use rsync, with mtime and not(!) operands and I’m able to keep 7 daily, 4 weekly, and 4 monthly backups. Runs every day, the rest of the backups are pruned in this rolling window.