Sync'ing/backup to Bubba using Unison - better than rsync?
Posted: 20 May 2008, 19:48
So I've been thinking of the best file-sharing and backup setup based around my new Bubba for the past week.
Seems like a lot of people are using Rsync to back up files from their desktops to Bubba, or to back up Bubba to an external USB hard drive.
But it occurred to me that since I'll be using my Bubba's /home/storage directly mounted via NFS on my "lite" devices (my eee-pc laptop when at home and my PS3 linux), it'd be nice to have a 2-way synchronisation setup so I can add new files to the Bubba's copy and have them synch'd to my external USB drive when I go on the road, but also vice versa.
The Unison project seems to fit the bill here (http://www.cis.upenn.edu/~bcpierce/unison/), and is available from the Debian Sarge repos, so I decided to give it a go.
The first trick to watch for is it needs exactly the same version on all your clients, Debian Sarge only offers 2.9.1 so make sure you ask for this specifically on your other (probably x86-based) machines. The second thing is user permissions, unison prefers to run as a user rather than root program (good in my opinion), but it means you'll need to get your permissions/users sorted on different machines so it can actually delete files.
It wasn't too hard to get set up though, and so far I'm testing synch'ing my ~3 Gig collection of photos between the Bubba and my External HDD mounted from my desktop. I deliberately made a few changes and it worked fine, once the initial synch is done a subsequent scan took only about a minute. I ran 'top' on the Bubba during this and although Unison was often taking 80% CPU (presumably it's doing lots of checksum calcs), it was using hardly any memory so isn't likely to crash it, though it could be a problem if you're running a non-trivial webserver on your Bubba too I guess.
Anyway next step is to configure synch'ing certain parts of my desktop's HDD as a cron job (or maybe straight after logon) in batch mode, with a list of conflicts emailed to me for review, which seems very possible. If anyone else is interested I could write that up as a proper HOWTO when I get it working. Phillip Guo also has a decent guide for setting it up - http://www.stanford.edu/~pgbovine/unison_guide.htm
Seems like a lot of people are using Rsync to back up files from their desktops to Bubba, or to back up Bubba to an external USB hard drive.
But it occurred to me that since I'll be using my Bubba's /home/storage directly mounted via NFS on my "lite" devices (my eee-pc laptop when at home and my PS3 linux), it'd be nice to have a 2-way synchronisation setup so I can add new files to the Bubba's copy and have them synch'd to my external USB drive when I go on the road, but also vice versa.
The Unison project seems to fit the bill here (http://www.cis.upenn.edu/~bcpierce/unison/), and is available from the Debian Sarge repos, so I decided to give it a go.
The first trick to watch for is it needs exactly the same version on all your clients, Debian Sarge only offers 2.9.1 so make sure you ask for this specifically on your other (probably x86-based) machines. The second thing is user permissions, unison prefers to run as a user rather than root program (good in my opinion), but it means you'll need to get your permissions/users sorted on different machines so it can actually delete files.
It wasn't too hard to get set up though, and so far I'm testing synch'ing my ~3 Gig collection of photos between the Bubba and my External HDD mounted from my desktop. I deliberately made a few changes and it worked fine, once the initial synch is done a subsequent scan took only about a minute. I ran 'top' on the Bubba during this and although Unison was often taking 80% CPU (presumably it's doing lots of checksum calcs), it was using hardly any memory so isn't likely to crash it, though it could be a problem if you're running a non-trivial webserver on your Bubba too I guess.
Anyway next step is to configure synch'ing certain parts of my desktop's HDD as a cron job (or maybe straight after logon) in batch mode, with a list of conflicts emailed to me for review, which seems very possible. If anyone else is interested I could write that up as a proper HOWTO when I get it working. Phillip Guo also has a decent guide for setting it up - http://www.stanford.edu/~pgbovine/unison_guide.htm