I have a Debian GNU/Linux server with a large raid 5 disk. I recently upgraded my RAID disks and suddenly have ample free space. I put OS X 10.5 on my MacBook back in November but didn’t have enough space at the time to try out Time Machine. Getting Time Machine to work with my Linux server was annoyingly hard–the default Debian server doesn’t support Leopard out of the box and Time Machine itself doesn’t support non-apple file shares. Not to mention Time Machine seems slow and completely bug ridden (I’m generally unimpressed, but when it finally works it’s nice). Anyway, here’s what I had to do to get it to work.
Step 1: rebuild netatalk with ssl support
Apple, starting with Leopard, won’t connect to appleshare servers that don’t support SSL (encryption). This is a good thing, really. What is annoying is that Debian doesn’t ship their netatalk package with encryption enabled (there’s apparently some sort of licensing mismatch and they’re fairly pedantic about those things–witness “iceweasleâ€).
If you are familiar with Debian packages, the key thing is rebuilding the netatalk package like this:
DEB_BUILD_OPTIONS=ssl debuild
If you are not familiar with Debian packages, first google for “debian ssl netatalk†or “ubuntu ssl netatalkâ€. There’s a bunch of people out there with instructions. They basically boil down to:
apt-get source netatalk sudo apt-get build-dep netatalk sudo apt-get install cracklib2-dev cd netatalk-2.0.3 DEB_BUILD_OPTIONS=ssl debuild sudo dpkg -i ../netatalk-*.deb
At this point you should be able to mount an your Linux server on your Mac.
Step 2: Getting The Mac To See The Server
Time Machine, for some stupid reason, doesn’t seem to want to see network shares from non-Apple servers. There’s a well known secret preference (how’s that for an oxymoron) that you have to set from Terminal:
defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
You can make sure it’s set with this command:
defaults read com.apple.systempreferences TMShowUnsupportedNetworkVolumes
It should print “1″ if it is enabled. With that set, you should now mount the drive you wish to back up to with Finder, then go to Time Machine’s Preferences. If all is well then your network drive will show up when you press the “Change Disk…†button.
If the drive does not show up then you are going to have to mess around. I had a terrible time getting it to show up on a computer with 10.5.2 installed. One thing to try is adding a file to the root of the network share called “.com.apple.timemachine.supportedâ€. Also try deleting the .0016cbcbc4c8 file in the root of the share (that number is for my computer–your computer will have a different number).
Step 3: Try a small backup
Go into Time Machine preferences and hit the “Options…†button. Keep adding excluded directories until only a small part of your disk is set to back up. Right click on Time Machine in the Dock and select “Back Up Nowâ€. If all goes well then Time Machine will chug along for a while “preparing†(going through your whole disk sizing everything up), then doing the actual backup, then “finishing†(Deleting interim backups–if you have a lot of these then finishing can take a really really long time).
If it gets an error then you might need to create the sparse bundle for it.
An aside–sparse bundles and Time Machine
A “sparse bundle†is a new format of disk image added in OS X 10.5. This follows a long line of disk image formats from the old “.img†of System 7 to “.dmg†introduced in OS X.
Sparse bundles are actually pretty neat. They are almost the same as “sparse image†“.dmgâ€s from previous versions of OS X. Traditional disk images allocate all the room up front. So if you create a 10GB disk image then it would create a 10GB file on your disk.
The sparse image is different–if you create a 10GB sparse disk image then the file that is created on the disk is actually much smaller, basically just the size of the HFS+ filesystem. When you add files to the disk image then the .dmg file itself grows to accommodate new data.
Sparse bundles work the same way except that instead of one gargantuan 10GB file (assuming you fill up the whole disk image), the data is spread out in a large number of small “bandsâ€. Each band is a file that is a specific size (8MB for Time Machine sparse bundles) that holds just one little section of the disk image’s data.
Why is this good? I’m glad I asked! It’s particularly nice when you are syncing the image between 2 computers. Say I mount a normal disk image and change one letter in a TextEdit document. If I try to sync the disk image to another computer I basically have to copy the whole thing over (barring some cool rsync-like behavior). With a sparse bundle, I probably have to only copy one or two bands of data across. According to what I’ve read, Apple create the sparse bundle format so that File Vault was more compatible with Time Machine (it can back up bands instead of entire disk images).
Anyway, when you back up to an external FireWire or USB disk, Time Machine backs up directly to the disk. When you back up to a network disk, Time Machine puts a sparse bundle disk image on the network share and then backs up to the disk image. This theoretically allows them to back up to a network filesystem that by itself doesn’t have the necessary capabilities for Time Machine. NFS, or SMB, for instance should be able to be able to host Time Machine backups with no issues–I haven’t tried this though.
The problem is that OS 10.5.2 seems to have a bug where you can’t always create a sparse bundle on a network share. If you try to do this with Disk Utility you will get an “operation not supported†error.
The solution is to create a sparse bundle on your disk that is the size of your network backup disk and then copy it over. My backup disk is 2.25 TB, so I’d create a 2.25 TB sparse bundle disk image. Once you copy it over to the network drive you can delete it from your local disk. I used this technique to get my dad up and running with his shiny new Ubuntu server.
By the way, for this to work you have to name the sparse bundle disk image correctly, which takes a little trick. Start a Time Machine backup and look in your network disk. Time Machine will have started creating a sparse bundle on the disk. Copy the name of this sparse bundle before it gets the error and deletes it on you. Mine is called “black_0016cbcbc4c8.sparsebundle†(my computer name is “blackâ€). Either create the sparse bundle with that name or rename it once you’ve created it.
Other issues
My computer behaved very, very strangely before I got everything running smoothly. One thing it did was stick in the “preparing†phase for huge periods of time (24 hours, sometimes). I’d just stop the backup and let it try again later. After a couple tries it would mysteriously work.
Another thing it did was get confused with the progress. It would start up a backup and say something reasonable in the progress bar (0GB completed out of 30GB). Everything moved along but when it hit the “end†it would suddenly start incrementing the total size: “32GB completed out of 32GBâ€. And it would just keep growing and growing. It would reach ludicrous numbers sometimes, saying “2TB completed out of 2TB†while backing up my 250GB disk. Sometimes it would complete on its own and sometimes I just stopped it after 24 hours.
Interestingly, both of these strange symptoms might be related. I captured both of these situations using 10.5’s dtruss and found that in both cases if was seemingly looping through my disk multiple times. I would see the same files and folders being iterated through over and over again, like there was something wrong with my filesystem. I did a fsck with Disk Utility, though, and it reported that everything was fine.
So I think both of those issues are just stupid bugs in Time Machine (or one stupid bug hitting in two places). It’s times like these when I curse Apple’s crappy proprietary software. If this were a linux app I’d just download the source and fix it myself. But the Bastards at Apple think it’s better for Time Machine to be a %$#@! black box. Which is OK when it works, but really sucks when it doesn’t work. </rant>
Step 4: Slowly remove excludes
To get around these issues I would alternately let it run for huge long periods of time and then stop it if it were running for more than 10 minutes. I also excluded almost all my disk in the Preferences (as I mentioned before) and then slowly removed exclusions until my whole disk was being backed up.
I’m at the point now where Time Machine just mounts up my network drive, backs up and only takes a couple minutes to finish. Getting there was slow (it took me about a month to get to this point!!!) and even though it is fairly stable now, I still can’t help feeling like Time Machine is a buggy pile of crap.
Some helpful links: