The buzz is building around Amazon Web Services as an application platform. Don MacAskill has been using AWS’ S3 storage service for SmugMug, and according to Jeremy Zawodny, will be talking about it at this year’s ETech conference. Jeremy and others have been experimenting with S3 as storage backup for desktop and laptop systems. Even Dave Winer is experimenting with S3.
When designing GigaVox Audio Lite (currently in by-invitation-only alpha test) I decided to use not just S3 (storage) but essentially all of AWS’ services for our infrastructure. This graphic should give you some idea of what we’re doing:
In addition to S3 (storage) I’m using the Elastic Compute Cloud (EC2) which allows us to instantiate servers on-demand. If we need a dozen extra servers for two hours of transcoding, we just fire them up for as long as we need them (perhaps a few hours) then shut them down. And that can be automated based on the load on the current servers. Dynamic computing power on demand. The dream has been realized.
To communicate among our servers and with the external systems we also make heavy use of the Simple Queue Service (SQS). There are actually many more queues than shown in the diagram which is somewhat out of date.
In asking around, it appears we have one of the most complex and sophisticated systems built to date using all three AWS services. It is infinitely scalable, extremely reliable and costs very little until we need it. And our system administrator doesn’t have to carry a pager. All those servers are someone else’s problem. No hardware or software to buy, maintain, upgrade, etc. It has been a lot of fun to actually make use of some of the asynchronous message-based architecture ideas in my book, Loosely Coupled. Hey…this stuff actually works! Sweet. Congratulations to Amazon for brilliant vision and execution.
If you want to see it in action for the production and publishing of your podcast, send me an email, tell me about your podcast, and I’ll send you back an invitation to the alpha test.
Blogarithms 
“all three AWS services…”
Amazon offers seven services, not three. You left out Amazon E-commerce Service, Amazon Mechanical Turk, Alexa, and Amazon Historical Pricing service..
LikeLike
I’m myself using S3 for quite some time — other than a stray slow response time once or twice, the service has been amazing. And even if the resp time is slow, Amazon folks are very quick to rectify it.
the amazon WS is just brilliant and a life-saver when it comes to startups who want to focus on execution and business rather than the headache of dealing with storage, backups and stuff like that.
slideshare.net also uses amazon s3 to host the flash based ppts.
LikeLike
Awesome setup, Doug. You’re one of the few out there really taking full advantage of Amazon’s offerings. For the curious, Basejumpr.com is another startup using S3, SQS, and EC2 in conjuction. I can’t wait to see what else gets built on top of Amazon’s infrastructure.
LikeLike
A while back I had built this: http://www.francisshanahan.com/robocast
with thoughts of perhaps developing it into a service. It looks like there are some parallels there but you have taken it to the next level with Gigavox.
Nice job.
LikeLike
I use S3 too for streaming videos over HTTP. you can check out the servcie at KONKAN TV. Apart from the ocassional failed SOAP / REST file push requests and ocassional CPAINT errors and/or streaming speeds, it’s been an awesome ride so far…
LikeLike
mduaoffrosmaofgoerosmonvosmogosnolvoslovnms
ncvosnrpos
LikeLike
Do you know where can I find the Amazon architecture? I mean, servers, and Databases.
LikeLike
Hi Doug, i was listening to your video on Amazon Web Services. Thanks for sharing the info.
I wanted to follow up on your decision to use the external Database server. Can you provide some detail on why you did not choose to use one of the images as a database server.
regards
LikeLike
The way that you break down the structure makes it much easier to understand. Great job.
Rick Marnon, Howell
http://www.oaklandlivingston.com/howell.html
LikeLike
what standards or lack of, were there when you were beginning to use Amazon’s web services?
LikeLike
Doug – where’s the graphic of the architecture?
LikeLike
Hi, I have a question about the real world usabitity of the Amazon Web Services. Could I run windows servers in windows vm clusters on the Amazon Elastic Compute Cloud (Amazon EC2) Windows Server running SQL, Office Middle Ware, VOIP… like a real hosted utility grid?
Can’t find any data on people using windows server os’s with it?
Thanks
LikeLike
Hi there, the graphic doesn’t show up any more 😦
Any way I can have a glance at it? I’m really interested in how it works with Amazon s3!
LikeLike
amazons architecture is totally different from others there way of building in entirely different and i coulnt follow the block diagram above
LikeLike
The architectural style of the amazons are totlly different to the rest of the world and the blog diagram provided is very informative thanks.
LikeLike
police officers to the immigration service and recruiting a further 360 new staff to boost efforts against illegal immigration.
LikeLike
Yea,i agree with Larry
LikeLike
Great post. Will read on…
LikeLike