backup

I use Media Temple for hosting of websites and Subversion repositories. The quality of service they provide is solid (especially considering the price) and the ability to SSH into the server and dig around at the command line is a huge plus. Unfortunately one of the areas they are lacking in is backups.

Media Temple provide a backup service but it is very limited in terms of capacity and granularity. Their backup function also does not provide any Subversion repository dump support which is crucial if you have ever experienced the pain of recovering from a Berkeleydb corruption.

Below is a script I use to backup my web domains, databases and Subversion repositories to compressed tar archives. Once these have been generated they are uploaded to the Amazon S3 storage cloud to create an off-site backup of off-site resources. The ability to backup directly from Media Temple to Amazon is great because it removes the middle-man (me) and maximises bandwidth usage. Pulling a tonne of data from the United States to New Zealand just to send it back again is certainly not an economical use of Internet resources.

I have written previously on how cool the little NSLU2 is as a customisable NAS device. I have set mine up as a little backup device, it silently backs up my server files (using rsync), creates tar files from all the files and then presents these archives to Retrospect on my Mac for backing up to external media and taking off-site. Unfortunately the default 'ls' and 'tar' programs that come with the Unslung distribution do not support large file sizes or long filenames to fix this problem download the far more up-to-date versions using ipkg:

After having my fair share of backup dilemmas and stress Tim Bray's post at least gives hope that I am not alone. It is strange that there really are not that many decent backup products considering how simple the task really is.

The most effective backup system I have used is the one I wrote myself for Linux servers. It is simple, flexible, non-proprietary and most importantly it seems to work without hassle. It comprises of a bash script and some configuration files.

The bash script compresses gzipped tar archives onto a removable hard drive. In most cases I use two removeable drives for redundancy. Typically all user data is stored in the /home directory with a few special directories: media (music/movies), temp and data shared amoungst the users. The /home/data directory holds most of the office data and as such the backup script breaks archiving of this data into smaller pieces in order to facilitate easier restoration.
The backup script stores multiple backups on disk to ensure that a corrupted backup does not lead to significant loss of data. The backup usually runs every night with the removable disks swapped during the day. On completion (or failure) of a backup an automated email is sent out to interested parties.