Remote, synchronized backup
Saturday, January 29th, 2005It all started when Jodi told me she accidentally deleted a mail in her mailbox. I remember setting up her mail client on her powerbook as a pop account, and enabling her to permanently delete mail. I decided to see how l33t I really was, and went into her mail directory on the server. I decided to try to undelete the file, and used grep to search the partition. I read somewhere you could do this, so I decided to search the last 1.8MB on the partition for the phrase “jodibell.com.” Not a good idea. For some reason, or somehow, the command locked up all processes. I’m talking: ssh, httpd, ncftpd, EVERYTHING. There was no way to stop the command, and unfortunately, no way to reboot the server remotely. I waited for a while, but then got really worried. You know, that sinking feeling when you start to imagine the worst possible outcome: what if my server was really hosed up good and I lost everything? When was the last time I backed up? How much would I lose!?!?
Well, to make a long story short, I eventually called my dedicated hosting provider, Interland, who are great, BTW, and they simply rebooted the box. She was up within 20 minutes of the ordeal. Yes, there goes my 99.9% uptime, and my 180+ days of continuous uptime with no reboots, but alas, at least she’s up and back purring….
So, to answer the question of when the last time I backed up was - well, let’s just say it has been too long.
I started searching for some backup scripts or even a backup program, but then decided to build my own. I am using a script running daily/weekly (we’ll see how it goes) in the crontab that runs rsync on several directories, including the entire seebq.com virtual host directory, to synchronize with my linux server at home - the mythtv-server. I have heard that rsync-diff is the way to go, but I’ll try that some other time. I also grabbed the etc directory and then proceeded to whip out some mysqldumps (since my movabletype is storing data in a mysql DB) and rsynced those too.
I had some difficulty adding the ssh key so it would automatically trust my mythtv-server at home, but then found out about running processes under the ssh-agent - and fixed it.
So the gist of my script does:
rsync --delete --compress --archive --rsh=ssh $directory_to_backup backupuser@$remote_host:backups/
And to get the mysql dumps, a quick:
mysqldump --opt -u $dbuser -p$dbpass $database > /root/mysqlbackup/mysql_database.bak.dump
and then I rsync the mysqlbackup directory, after performing several mysqldumps on the various directories.
The hardest part is setting up the automatic ssh login. So - let’s just hope, like every good backup, that I never have to use it.
References:
http://rimuhosting.com/howto/rsyncbackup.jsp
http://www.cs.indiana.edu/Facilities/FAQ/Security/openssh.html
http://rimuhosting.com/howto/mysqlbackup.jsp
