How Twitter Integrates New Architecture

I've been experimenting with the Cassandra datastore lately and came across an interesting interview with Ryan King discussing Twitter's use of Cassandra.  I thought this tidbit was worth sharing even if you don't care about Cassandra:

A philosophical note here — our process for rolling out new major infrastructure can be summed up as “integrate first, then iterate”. We try to get new systems integrated into the application code base as early in their development as possible (but likely only activated for a small number of people). This allows us to iterate on many fronts in parallel: design, engineering, operations, etc.

He also describes in detail how they used that philosophy to migrate Twitter status updates (the toughest part of their app to scale) to Cassandra.  This also reminded me of Flickr's Feature Flags ... all of which can be used to support continuous deployment, something I've become very interested in after encountering it in the lean startup literature.

I'm planning to writeup my findings on using Cassandra with Ruby sometime soon if there's interest.

Scaling Rails in the Cloud

Yesterday at SXSW Interactive I gave a talk called "Scaling Rails in the Cloud", about our experiences building OtherInbox using Amazon Web Services. My slides are below. When I wrote this presentation, I really came to the realization that Rails has hardly ever been the problem for us, scaling-wise. The framework has served us very well; all of our problems have stemmed from we the developers not understanding our tools as well as we could have, or not thinking things through. So the actual Rails content is light, but that's because Rails fits right into a cloud environment and we didn't have to do anything special to make it work.

Scaling Rails Applications In The Cloud

View more presentations from subelsky.