HistoryI started with Time Machine talking to my Linux box with netatalk. This worked fine until one day Time Machine told me it needed to back up everything all over again.
Then it started to tell me this almost weekly. Apparently when this happens, the existing backup is corrupt and all that precious data is at worst irretrievable or at best tedious to retrieve. These are not attributes I want associated with my backup solution.
Then I did an rsync over ssh to my Linux box. This is fine exception that it lacks all the special permissions, resource forks, acls, etc, etc that are hidden in a Mac filesystem.
Then I tried SuperDuper! backing up to a directory served up via netatalk mounted via afp:// on OS X. This worked, but was mind numbingly slow. Also, it would mean I'd have to pay for a tool if I wanted to do incremental backups. This gets expensive as I also back up a few friends OS X laptops on my Linux file server.
I tried SuperDuper! backing up over Samba, but hdiutil create apparently doesn't work over Samba. Workarounds all needed the purchased version of SuperDuper!.
There's *another* work around for SuperDuper! where I can use MacFUSE and sshfs, but the MacFUSE author has abandoned the project and recommends people not to use it.
Ultimately, the goal is to make a sparsebundle HFS+ disk image, put it on a Samba mounted share and rsync my data over to it. You'd be surprised how many niggly bits there are for this.
First, I grabbed the 3.1.x version of rsync from Homebrew - install Homebrew as per the directions there, then run:
brew install https://raw.github.com/Homebrew/homebrew-dupes/master/rsync.rb
If you've been digging through voodoo magic, then you'll be happy to hear this version of rsync has the all the rsync patches you'll read about (like --protect-decmpfs).
Nobody needs another out of date blog entry explaining how to setup Samba. Follow some other guide, make sure Samba starts automatically and use smbpasswd to create an account.
I recommend using the name of the machine being backed up as the account name. I'm calling that machinename for the rest of this post.
Make sure you can mount this share on OS X via smb:// ( Finder > Go > Connect to Server... ). Make sure you can 1) create a file, 2) edit and save the file, 3) delete the file. I'm going to assume you've mounted this share at /Volumes/Machinename
Now lets make something for us to backup into. Figure out how big the disk is on the source machine (we'll assume 100g) then run:
hdiutil create /tmp/backup.sparsebundle -size 100g -type SPARSEBUNDLE -nospotlight -volname 'Machinename-Backup' -verbose -fs 'Case-sensitive Journaled HFS+'
Yes, you're creating it in /tmp, this is to work around hdiutil create not liking Samba.
Next you'll want to copy this sparse bundle onto your Samba share:
cp -rvp /tmp/backup.sparsebundle /Volumes/machinename
This will copy a bunch of files and should be successful without any warnings. Now lets mount this sparse bundle:
hdiutil attach /Volumes/machinename/backup.sparsebundle -mount required -verbose
You should now have /Volumes/Machinename-Backup mounted on your system. Fun story, OS X recognizes that this disk image is hosted off the machine, so it mounts this disk image with "noowners" (see mount man page). That's going to be a problem for our backup, so we need to tell OS X it's okay to use userids normally:
sudo diskutil enableOwnership /Volumes/Machinename-Backup
There are a handful of files which recommend to be excluded:
Store this in a file somewhere. I stored mine as exclude-from.txt in /Volumes/Machinename
Okay, now we're ready to run rsync. I think the correct arguments to rsync are: -aNHAXx --nfs-compression --protect-decmpfs --itemize-changes --super --fileflags --force-change --crtimes
So, we run:
rsync all-those-args-above --exclude-from=/Volumes/Machinename/exclude-from.txt / /Volumes/Machinename-Backup/
When The Backup Is Done
This will take a little while. When it's done, you can then bless your backup so it could be booted:
sudo bless -folder /Volumes/Machinename-Backup/System/Library/CoreServices
Then you can umount your backup:
hdiutil detach /Volumes/Machinename-Backup
Periodically, and after your first run, you should compact down your sparsebundle disk image:
hdiutil compact /Volumes/Machinename/backup.sparsebundle -batteryallowed
You can now log into your Linux server and tar up the backup (apparently XZ > bzip2 for compression size).
tar Jcvf machnename-backup.tar.xz backup.sparsebundle
Depending on the size of that tarball, you could upload it to Google Drive, Drop Box, etc. Before you do, you'll probably want to encrypt it. I used OpenSSL:
openssl aes-256-cbc -a -salt -in machinename-backup.tar.xz -out machinename-backup.tar.xz.aes-256-cbc
Many of these steps will take *hours* if you have a lot of data, so you may consider just backing up parts of your system more frequently and doing your whole system once every-so-often.
Why This Is Nice
One thing I really love about this setup is that each piece of the puzzle does one thing and does it well. If Samba is too slow, I could go back to netatalk. The sparsebundle disk image hosts HFS+ properly with all its OS X-specific voodoo and rsync's job is to copy files. If there's a better file copier, I could drop that in.
I left a lot out. I know. I'm kind of expecting you to have a rough idea of how to get around OS X and Linux, figure out how to put most of the above in a shell script, decide when to do backups, how to store those tarballs, etc. Hopefully though this will help someone that just needs some of the key ingredients to make it work.