Lean startup tools for Rails apps

A few months ago I was invited to dinner with the Geeks on a Plane crew when they stopped in Washington, and had the opportunity to meet one of my heroes, Eric Ries, author of the Startup Lessons Learned blog. His descriptions of lean startup techniques and philosophies have had a big influence on the way I design and build software.

Eric asked me what specific tools Rails developers could use when building a lean startup. Here's a revised version of what I emailed in response, garnered while building OtherInbox and other Rails apps:

• First and foremost, I find Rails itself to be very useful for building a product for a lean startup; the expressiveness of Ruby and the conventions of Rails help developers more quickly build a minimum viable product. I'm still able to surprise people who have worked with me for awhile with how quickly I can turn their feature ideas into
  something they can play with and show customers.


• For continuous integration, like many Rails shops, we use CruiseControl.rb - it's easy to install and customize, but I don't think people are especially enamored of it. But it works. Two very promising alternatives to rolling your own CI server are Devver and RunCodeRun. I've tried both but our app has grown too complex for either of them (we use a lot of gems and have a pretty customized test environment). For Github users, RunCodeRun exposes a post-commit hook so the tests run automatically after every commit which is pretty useful.


• Speaking of Github, I'd love for them to implement pre-commit hooks, because then you could prevent developers from making commits any time the build is broken, which prevents people from ignoring problems in a large code base and just tunneling on building their one little feature. Right now if you want to implement this, you need to setup your own source control repository that can talk to the continuous integration system.


• To monitor exceptions, we use the excellent Hoptoad app. This tool adds a lot of context to bug discussions, because nonprogrammers can reference the exact Hoptoad URL containing all of the information a programmer needs to fix a bug (including backtraces). Because it has an RSS feed, one could also use "Unresolved Hoptoad Errors" as a metric influencing the continuous deployment system. If a release goes out and suddenly there are new errors, that's a sign that the batch was bad and needs to be reverted. There's a drop-in Rails plugin but this service could be used with any language.


• There are two A/B testing plugins for Rails: 7 Minute ABs and A/Bingo. I haven't used either and am not sure there are any better than rolling your own. Update: Charlie Park in the comments mentions the new plugin Vanity which came out just after I wrote this.


• There are several good unit testing frameworks for Rails, each with their own devotees. What comes with Ruby and Rails is plenty good enough but some people like the greater expressiveness afforded by things like Rspec and Shoulda. For my next product I'd like to try Shoulda because it uses Ruby's built-in test facilities (making for one less dependency to worry about, as is the case with Rspec) and because I really like the examples I've seen on the Thoughtbot blog.


• We love Cucumber for higher level testing. When coupled with Webrat, which simulates a real browser, you get a pretty nice mechanism for exercising the entire app in a test environment before deployment. Webrat even has a Selenium adapter, so you actually can run your tests inside of a browser to make sure all is well before deploying a change.


• This isn't Rails-specific, but we use SimpleDB for dumping a lot of important metrics data, like performance measurements, which we then turn into metrics. This keeps us from having to hit our database too much. The RightAWS gem makes interacting with SimpleDB pretty easy.


• For our metrics we mainly use Graphite, the graphing and analysis system open-sourced last year by Orbitz. Our whole team is falling more in love with it every day.


• New Relic has a nice plugin for Rails that reports all kinds of useful data to their service, which could become part of a cluster immune system for Rails apps. They measure average responsiveness and compute an aggregate statistic called Apdex which can be a good indicator of site health (at minimum, any significant changes to Apdex should be investigated, and they will email you alerts about such changes).


• Capistrano is still the standard way most developers are deploying Rails apps (but also check out Vlad the Deployer), but I think there's an opportunity for someone to build a Ruby library to facilitate continuous deployment that would work better than Capistrano. It would definitely be built using Rake like Vlad does.


• Update: I haven't used Heroku, but as Ryan points out in the comments it definitely belongs on this list as a compelling, get-up-and-running fast deployment platform.