Amazon for Infrastructure-on-Demand

The buzz is building around Amazon Web Services as an application platform. Don MacAskill has been using AWS’ S3 storage service for SmugMug, and according to Jeremy Zawodny, will be talking about it at this year’s ETech conference. Jeremy and others have been experimenting with S3 as storage backup for desktop and laptop systems. Even Dave Winer is experimenting with S3.

When designing GigaVox Audio Lite (currently in by-invitation-only alpha test) I decided to use not just S3 (storage) but essentially all of AWS’ services for our infrastructure. This graphic should give you some idea of what we’re doing:


In addition to S3 (storage) I’m using the Elastic Compute Cloud (EC2) which allows us to instantiate servers on-demand. If we need a dozen extra servers for two hours of transcoding, we just fire them up for as long as we need them (perhaps a few hours) then shut them down. And that can be automated based on the load on the current servers. Dynamic computing power on demand. The dream has been realized.

To communicate among our servers and with the external systems we also make heavy use of the Simple Queue Service (SQS). There are actually many more queues than shown in the diagram which is somewhat out of date.

In asking around, it appears we have one of the most complex and sophisticated systems built to date using all three AWS services. It is infinitely scalable, extremely reliable and costs very little until we need it. And our system administrator doesn’t have to carry a pager. All those servers are someone else’s problem. No hardware or software to buy, maintain, upgrade, etc. It has been a lot of fun to actually make use of some of the asynchronous message-based architecture ideas in my book, Loosely Coupled. Hey…this stuff actually works! Sweet. Congratulations to Amazon for brilliant vision and execution.

If you want to see it in action for the production and publishing of your podcast, send me an email, tell me about your podcast, and I’ll send you back an invitation to the alpha test.