A few weeks ago I had the misfortune of having 2 hard drives die on me simultaneously on blackwidow. I have an LVM2 setup with no redundancy, so this means I pretty much lost everything. Fortunately I make monthly hard copies of my configuration and daily backups of everything important, but those backups are to an old and slow DDS3 drive. I knew this was gonna be an all-day thing.

I replaced the 2 dead drives with 2 brand new 250GB SATA drives (I have officially broken the 1TB barrier . Re-creating the LVM volumes was pretty straight-forward and predictable. The issue came when I had to restore files. This became a PITA. I had always relied on the official 'Gentoo LiveCDs'_ for rescue disks. However, I then discovered that my copy did not have the SCSI tape driver. I had my LVM volumes visible and usable but could not read my amanda tapes.

My next try was to use an old copy of KNOPPIX that I had floating around. The KNOPPIX CD was scratched up. It was readable, but it was slow reading it and some things just didn't run because my CD drive could not read the area where some programs were located. Fortunately KNOPPIX saw my tape drive and had the mt command, but there were two other problems. The first, minor, problem was that there were no Amanda tools on the KNOPPIX CD. That was okay because Amanda backup images are basically tar archives with a header in front. So dd with the skip option worked on that. Unfortunately the area on the disk where dd was located must have had scratches because it would take my drive forever to load dd. The second issue was more critical: my KNOPPIX CD did not recognize LVM volumes. The only sizeable filesystem I had that was not on LVM was my root, but it was not large enough to hold all my backups. The only way I could make things work with what I had was to, one by one, restore an image to my root partition in KNOPPIX, then reboot with my Gentoo LiveCD and restore the image onto the target LVM volume. This took forever rebooting, positioning the tape, restoring, rebooting, over and over. The fact that my KNOPPIX CD was scratched up and so took longer to read did not help either.

After an exhausting day and a half I finally got my system back to a bootable, usable form. I vowed it would never again be this difficult to recover my system. In order to do this I made the following changes. First, I have different kinds of Live CDs and multiple copies of them. I also store them instead of just having them lying around so they don't get scratched. But the most important thing I did was I got rid of the tape drive. Tape drives are just slow (unless you can afford a fast one) and anyway disk space is cheaper and faster. Why wasn't I doing disk-to-disk backups sooner? I decided on getting a hard disk enclosure and another drive for backups. I got a nice Penguin Gear enclosure and an 80GB SATA drive. I could have gotten a bigger drive but 80GB is already larger than the capacity of all my tapes put together so I figured why waste all that space on a dedicated backup drive.

I had heard a lot of good things about BackupPC. I decided I was gonna dump Amanda, which is more suited for tape backups, and go with BackupPC which is designed more for disk-to-disk. I emerged backuppc and tried to configure it. Well, the configuration file which is basically a Perl script didn't give me a good feeling, but I finally was able to set it up with the help of the documentation. I then started the daemon. After that I tried "Talking to BackupPC" with the ''BackupPC_serverMSG'' command as in the documentation but none of their examples worked. I kept getting invalid command. Well, I figured I'd mostly be using the CGI interface anyway, so I tried viewing it with my browser. Server error. Apparently the CGI expects Apache to be running as the ''backuppc'' user. Well I need apache running as apache for other things. I tried to make the CGI suid but it's a perl script so of course that doesn't work with Linux.

I became more and more frustrated with BackupPC, so I decided to see what it took to do disk-to-disk backups with Amanda. I found the docs on their web site. It was actually a lot easier than I thought. All I had to do is create "virtual tapes" on my backup drive and tell amanda to use them. I was even able to use my existing Amanda configuration by removing the old physical tapes from the config and then adding the virtual ones. I ran a test dump and Amanda backed up my 3 systems to disk in a flash. Maybe it's just because I'm more familiar with Amanda than BackupPC, but to me it was so much easier with Amanda I regret not having tried it first. Restoring is easy as well.

So why am I talking about this? Well, I first want people to know that even a person like myself who considers himself disaster prepared can run into problems. It's always good to keep a checklist of what you need in case of a disaster and check it regularly to ensure that everything is working and in good condition. Secondly, with the price of disk space these days nearly everyone should have a system for making backups of important stuff. I can't believe I stuck with that old unsightly and slow DDS drive for so long.