Personal Backup on Amazon S3

Jeremy Zawodny is running some interesting tests and comparisons using Amazon’s S3 storage service for remote backup of his personal computers. Read the comments, too.
For me, a good backup solution has one more requirement: versioning. I wrote about the problem in my first book, Strategies for Web Hosting and Managed Services. I refered to the problem as Propagation of Corrupted Data:

Consider what happens if the original version of  file becomes corrupted–perhaps due to faulty hardware or software–but the corruption goes undetected. The corrupted version of the file will be copied to the archive server at the next backup interval. That’s okay, as long as there’s still at least one other, older archive that contains an uncorrupted copy of the file. Otherwise, the corrupted copy will replace the only remaining uncorrupted version.

There are three backup use cases. First is the one we think about most often, catastrophic failure, in which you probably want to recover an entire drive. This is infrequent but high value. Second, much more frequent, is recovering a file because of human error. Like everyone else, I sometimes delete a file (or directory — aargh!) by accident and I need to get it back.

But what happens when a file is damaged for one reason or another? Most backup systems will simple replace the good copy of the file with the “new” damaged copy, so there’s no way to recover the good copy. If you run  rsync or other backup once a day, you’ve got a 24-hour (or less) window to discover the corruption and recover from your backup. That’s why I always use rotating backups. I don’t overwrite one backup with another. I create a new copy or image. I’ll then keep a weekly, monthly and annual version of everything, so there’s a good chance I can recover pre-corrupted versions. It’s not as robust as a fully versioned system (like svn) but it has saved my butt.

My ideal remote-backup soultion would be (a) automatic, (b) unattended, and (c) one that included at least some form ove versioning to keep different versions of modified files.

Podcast Expo 2006

This year’s PME is over and was another great success. Tim Bourquin and team are to be congratulated once again. The GigaVox Media booth was so popular, that I didn’t have a chance to attend any sessions other than my own, and I didn’t even walk past most of the other booths. I did get to spend a few minutes with Marvin Caesar, president of Aphex, one of my favorite audio-gear manufacturers. Paul Figgiani, Michael Geoghegan and I are big-time Aphex fans.

My first presentation was entitled Prepare for Success:
Overcoming the Challenges of Scale
. I thought it went very well, but unfortunately as a last-minute replacement, it wasn’t in the program or addendum, and I had an audience of a whopping eight (!) people in a room set for about 400. Ouch. My second presentation, The Secret Lives of MP3 Files, had a much larger audience. Maybe 100 or so. I warned people in advance that it would be very geeky, and it was.
The Levelator is already a huge hit. I finished uploading the Java UI code just after midnight. Twelve hours later we had more than 200 downloads, and the buzz on the exhibit-hall floor was great. People like Jake Luddington have already blogged about it. (You can digg Jake’s post.) Others even came by with their laptops, just to show us how The Levelator had solved some of their most-difficult audio problems. I think the word will be out soon, and a vast majority of podcasters (and broadcasters) will start using The Levelator.

After spending two months writing code for 60+ hours a week — yeah, sob for me 🙂 — it’s time for a break. Gonna take a few days off.