I have a Debian GNU/Linux server with a large raid 5 disk. I recently upgraded my RAID disks and suddenly have ample free space. I put OS X 10.5 on my MacBook back in November but didnâ€™t have enough space at the time to try out Time Machine. Getting Time Machine to work with my Linux server was annoyingly hardâ€“the default Debian server doesnâ€™t support Leopard out of the box and Time Machine itself doesnâ€™t support non-apple file shares. Not to mention Time Machine seems slow and completely bug ridden (Iâ€™m generally unimpressed, but when it finally works itâ€™s nice). Anyway, hereâ€™s what I had to do to get it to work.
Step 1: rebuild netatalk with ssl support
Apple, starting with Leopard, wonâ€™t connect to appleshare servers that donâ€™t support SSL (encryption). This is a good thing, really. What is annoying is that Debian doesnâ€™t ship their netatalk package with encryption enabled (thereâ€™s apparently some sort of licensing mismatch and theyâ€™re fairly pedantic about those thingsâ€“witness â€œiceweasleâ€).
If you are familiar with Debian packages, the key thing is rebuilding the netatalk package like this:
If you are not familiar with Debian packages, first google for â€œdebian ssl netatalkâ€ or â€œubuntu ssl netatalkâ€. Thereâ€™s a bunch of people out there with instructions. They basically boil down to:
apt-get source netatalk sudo apt-get build-dep netatalk sudo apt-get install cracklib2-dev cd netatalk-2.0.3 DEB_BUILD_OPTIONS=ssl debuild sudo dpkg -i ../netatalk-*.deb
At this point you should be able to mount an your Linux server on your Mac.
Step 2: Getting The Mac To See The Server
Time Machine, for some stupid reason, doesnâ€™t seem to want to see network shares from non-Apple servers. Thereâ€™s a well known secret preference (howâ€™s that for an oxymoron) that you have to set from Terminal:
defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
You can make sure itâ€™s set with this command:
defaults read com.apple.systempreferences TMShowUnsupportedNetworkVolumes
It should print â€œ1â€³ if it is enabled. With that set, you should now mount the drive you wish to back up to with Finder, then go to Time Machineâ€™s Preferences. If all is well then your network drive will show up when you press the â€œChange Diskâ€¦â€ button.
If the drive does not show up then you are going to have to mess around. I had a terrible time getting it to show up on a computer with 10.5.2 installed. One thing to try is adding a file to the root of the network share called â€œ.com.apple.timemachine.supportedâ€. Also try deleting the .0016cbcbc4c8 file in the root of the share (that number is for my computerâ€“your computer will have a different number).
Step 3: Try a small backup
Go into Time Machine preferences and hit the â€œOptionsâ€¦â€ button. Keep adding excluded directories until only a small part of your disk is set to back up. Right click on Time Machine in the Dock and select â€œBack Up Nowâ€. If all goes well then Time Machine will chug along for a while â€œpreparingâ€ (going through your whole disk sizing everything up), then doing the actual backup, then â€œfinishingâ€ (Deleting interim backupsâ€“if you have a lot of these then finishing can take a really really long time).
If it gets an error then you might need to create the sparse bundle for it.
An asideâ€“sparse bundles and Time Machine
A â€œsparse bundleâ€ is a new format of disk image added in OS X 10.5. This follows a long line of disk image formats from the old â€œ.imgâ€ of System 7 to â€œ.dmgâ€ introduced in OS X.
Sparse bundles are actually pretty neat. They are almost the same as â€œsparse imageâ€ â€œ.dmgâ€s from previous versions of OS X. Traditional disk images allocate all the room up front. So if you create a 10GB disk image then it would create a 10GB file on your disk.
The sparse image is differentâ€“if you create a 10GB sparse disk image then the file that is created on the disk is actually much smaller, basically just the size of the HFS+ filesystem. When you add files to the disk image then the .dmg file itself grows to accommodate new data.
Sparse bundles work the same way except that instead of one gargantuan 10GB file (assuming you fill up the whole disk image), the data is spread out in a large number of small â€œbandsâ€. Each band is a file that is a specific size (8MB for Time Machine sparse bundles) that holds just one little section of the disk imageâ€™s data.
Why is this good? Iâ€™m glad I asked! Itâ€™s particularly nice when you are syncing the image between 2 computers. Say I mount a normal disk image and change one letter in a TextEdit document. If I try to sync the disk image to another computer I basically have to copy the whole thing over (barring some cool rsync-like behavior). With a sparse bundle, I probably have to only copy one or two bands of data across. According to what Iâ€™ve read, Apple create the sparse bundle format so that File Vault was more compatible with Time Machine (it can back up bands instead of entire disk images).
Anyway, when you back up to an external FireWire or USB disk, Time Machine backs up directly to the disk. When you back up to a network disk, Time Machine puts a sparse bundle disk image on the network share and then backs up to the disk image. This theoretically allows them to back up to a network filesystem that by itself doesnâ€™t have the necessary capabilities for Time Machine. NFS, or SMB, for instance should be able to be able to host Time Machine backups with no issuesâ€“I havenâ€™t tried this though.
The problem is that OS 10.5.2 seems to have a bug where you canâ€™t always create a sparse bundle on a network share. If you try to do this with Disk Utility you will get an â€œoperation not supportedâ€ error.
The solution is to create a sparse bundle on your disk that is the size of your network backup disk and then copy it over. My backup disk is 2.25 TB, so Iâ€™d create a 2.25 TB sparse bundle disk image. Once you copy it over to the network drive you can delete it from your local disk. I used this technique to get my dad up and running with his shiny new Ubuntu server.
By the way, for this to work you have to name the sparse bundle disk image correctly, which takes a little trick. Start a Time Machine backup and look in your network disk. Time Machine will have started creating a sparse bundle on the disk. Copy the name of this sparse bundle before it gets the error and deletes it on you. Mine is called â€œblack_0016cbcbc4c8.sparsebundleâ€ (my computer name is â€œblackâ€). Either create the sparse bundle with that name or rename it once youâ€™ve created it.
My computer behaved very, very strangely before I got everything running smoothly. One thing it did was stick in the â€œpreparingâ€ phase for huge periods of time (24 hours, sometimes). Iâ€™d just stop the backup and let it try again later. After a couple tries it would mysteriously work.
Another thing it did was get confused with the progress. It would start up a backup and say something reasonable in the progress bar (0GB completed out of 30GB). Everything moved along but when it hit the â€œendâ€ it would suddenly start incrementing the total size: â€œ32GB completed out of 32GBâ€. And it would just keep growing and growing. It would reach ludicrous numbers sometimes, saying â€œ2TB completed out of 2TBâ€ while backing up my 250GB disk. Sometimes it would complete on its own and sometimes I just stopped it after 24 hours.
Interestingly, both of these strange symptoms might be related. I captured both of these situations using 10.5â€™s dtruss and found that in both cases if was seemingly looping through my disk multiple times. I would see the same files and folders being iterated through over and over again, like there was something wrong with my filesystem. I did a fsck with Disk Utility, though, and it reported that everything was fine.
So I think both of those issues are just stupid bugs in Time Machine (or one stupid bug hitting in two places). Itâ€™s times like these when I curse Appleâ€™s crappy proprietary software. If this were a linux app Iâ€™d just download the source and fix it myself. But the Bastards at Apple think itâ€™s better for Time Machine to be a %$#@! black box. Which is OK when it works, but really sucks when it doesnâ€™t work. </rant>
Step 4: Slowly remove excludes
To get around these issues I would alternately let it run for huge long periods of time and then stop it if it were running for more than 10 minutes. I also excluded almost all my disk in the Preferences (as I mentioned before) and then slowly removed exclusions until my whole disk was being backed up.
Iâ€™m at the point now where Time Machine just mounts up my network drive, backs up and only takes a couple minutes to finish. Getting there was slow (it took me about a month to get to this point!!!) and even though it is fairly stable now, I still canâ€™t help feeling like Time Machine is a buggy pile of crap.
Some helpful links: