I'm Starting a New Company, Improv Style

When people hear that I teach and perform improv theater, they often ask about the practical benefits of improv.  Different people take improv training for different reasons, but the improv lesson that's
helping me most as a software entrepreneur is the aphorism "Leap and the net will appear": if you stand on the sidelines during an improv scene, waiting for your scene partners to create an opportunity, or waiting for some brilliant premise to appear in your mind, you will fail.  Good things only come from taking risks, from opportunities you create for yourself.

In this light, I have decided to reduce the time I spend working on OtherInbox, which I cofounded three years ago, to build new products that may lead to new companies based here in Baltimore.  I was the
only remote developer working on OtherInbox, which is based in Austin, and as the local team gelled and our application became more complex, it seemed like a good time for me to try something new.  I'll still be involved in developing and maintaining OtherInbox code, but I'm now actively searching for other interesting problems to work on, problems I can solve from my home here in Baltimore.  It's time for me to practice what I preach here on this blog and what I teach to improv students.

I'm really proud of what the OtherInbox team has accomplished and I'm psyched to see what 2011 has in store -- if you're looking for a web development job in Austin you should definitely check our company out!

Speaking of taking big risks, my friend Josh Baer took a huge risk on me by inviting me to start OtherInbox with him.   Although we knew each other in college, I was very green and untested at the time as a professional developer.  I'm forever grateful for what I've learned firsthand from Josh about building a company and a software product from scratch, and for the chance I've had to build my reputation by blogging and presenting about the technology we use.

If you are experiencing some kind of life aggravation that you think could be solved with software, I'd love to talk to you!  No aggravation is too big or too little!  Email me at mike@subelsky.com.

The OtherInbox Pivot

We turned a sharp pivot this year at OtherInbox, shifting away from the product I initially built (now called Defender 1.0) to a newer, easier-to-adopt product called Organizer.  I had a lot invested in that product; I cut my teeth in the startup world while building it.  We worked hard to write the code and I even dressed up as a superhero at SXSW to promote it. I gave a lot of technical talks about the technologies we used, and I really loved the whole process, but the fact is we could not build a sustainable business with Defender.  


I'm really proud of us for pivoting to Organizer and don't feel the least bit bad about it (though I am sorry for anyone inconvenienced by the need to upgrade to our Google-hosted alternative, called Defender 2.0 - it really is a lot better for everyone in the long term, though).


Josh Baer gave a great explanation of how we reached this conclusion which I think really captures the decisionmaking and guesswork of entrepreneurship:

We had to pivot and focus our attention on Organizer, even though Defender was our first love.

It took too long

Nobody wanted to hear this. I certainly didn't want to admit that the "great idea" we started with was not going to be successful. We probably should have began this process six months earlier than we did but it was hard to swallow.

That put us in a situation where we had more than 500,000 users signed up for Organizer and less than 20,000 for Defender. Only about 1000 were paying $20/year.  Yet Defender was about half of our code base and 80% of our customer support issues. It was hard to put many resources into improving Defender because 95% of our users were using Organizer. Yet everyone on the team believes in Defender and uses it ourselves. It hurt the morale of the entire team to see a product stagnate from lack of attention. We needed a way to combine our resources so that all of the effort we put into the product can be benefit both Defender and Organizer users and so that we could deliver a more reliable service.

The good news is, Organizer is doing really well and we're making it better every day!  Check out the full post; it's a great case study!

Ruby for Startups at Lone Star Ruby Conference 2009

Recently Confreaks posted video from the Lone Star Ruby Conference talk I gave in August, called Ruby for Startups.

I've refined the talk a few times since then but it's still a good representation of my current state of mind about what I've learned about Ruby and about software design while building OtherInbox. Check it out if you'd like to see more! The slides are posted here.

It's Not Always Sunny in the Clouds

At RailsConf last week I gave a talk called "It's Not Always Sunny in the Clouds" where I shared all of our lessons learned with OtherInbox and cloud computing.  I tried my best to go beyond the hype and talk about our real world experiences.  The slides are below, but here's the raw list of lessons:

• “Everything needs to be automated”
  


• Autoscaling is the easiest part


• Think carefully about credential management


• You really could use internal DNS


• It's maybe not that cheap


• Launching servers is not that fast


• You will become dependent on “glue” services


• You will depend on a distant faceless provider


• Use DVCS


• You will spend a lot of time on monitoring


• Your logs will runneth over


• Write lots of “in-process tests”


• Snapshots are slow


• Rails will be the least of your worries


• Cloud services involve subtle-yet-massive tradeoffs


• SQS guarantees delivery at least once


• Queue lengths inaccurate for <>
  
• SQS not necessarily FIFO


• So you may not want a cloud queue


• SimpleDB optimized for writes, not reads


• You must code defensively


• There are no good "cloud sandboxes"


• Pay attention to MySQL timeouts


• "User account management is -not- ideal."


• You are locked-in to your provider


• Relational DB may not be the best choice


• Is there a benefit?
  
  
  
  • Changes the way you write code
  
  
  • You can start right away
  
  
  • Pretty awesome redundancy
  
  
  
  

It's Not Always Sunny in the Clouds

View more presentations from subelsky.

Today we launch OtherInbox!

Josh and I are in San Francisco at Techcrunch50.  Sometime between 1 and 5 pm Pacific time we'll be launching OtherInbox!  You can see the demo live at otherinbox.com.

What a great year it's been!  Thanks for everyone's help and support.

Speaking at Lone Star Ruby Conference

I will be speaking at the Lone Star Ruby Conference in September about how we use Ruby to deploy, monitor, and manage a cluster of servers running in the Amazon Web Services virtual cloud.   Below is a summary of what I'll be talking about.

In OtherInbox, almost every system administration task imaginable is carried out using Ruby, meaning we as developers can enjoy all of Ruby's expressive benefits and spend less time scripting the shell, writing cron tasks, or using other languages. Because we make fewer context switches from thinking in Ruby to thinking in other languages, we also reap a big productivity benefit.

Using Ruby throughout our cloud also means that porting the application to run in different production environments is a trivial task, because Ruby is the glue connecting the Ruby components together, thus all we require is a Ruby interpreter to deploy.

Two key Ruby technologies have matured in the previous 18 months which make it ideal for almost every layer of managing a cluster of servers:

• god.rb allows fine-grained process monitoring and daemon control (a la monit)
• rufus-scheduler enables Ruby-based scheduling (replacing cron, and providing a great facility for running daemons that must be executed on a recurring basis)

When combined with these Ruby workhorses, developers today can spend much more of their time writing Ruby code, and less time struggling with the vagaries of their production environment:

• Ruby standard library utilities (File, FileUtils, etc.)
• Rake
• Capistrano (for any kind of remote application, not just Rails)

The talk will also include a discussion of using several different AWS gems to make cloud computing simple, by illustrating the use of Amazon's S3 and SQS services to distribute asychnronous work and handle communication between servers.

(cross-posted from the OtherInbox blog)

RailsConf 2008 Recap

I wrote up a few articles on the OtherInbox blog about my experiences at RailsConf 2008:

• Friday recap (where I got to meet Geoffrey Grosenbach and sign some books)
• Saturday recap (where things get technical)
• Birds-of-a-Feather session recap (where I get to talk with cool people)

The best blow-by-blow coverage so far is from Drew Blas.  Of the ones I attended or heard the best feedback about, I most strongly recommend looking at the slides for these: 

• Surviving the Big Rewrite
• Fast, Sexy, and Svelte: Our Kind of Testing
• Refactoring Your Rails Application
• CI for the Rails Guy (or Gal)
• Facebook Development on Rails (no slides posted yet)

ORDER BY null kills MySQL filesorts dead

I spent some time today optimizing OtherInbox.  As our private beta expands, we are starting to see heavier usage, and so it's time to revisit some of my beloved SQL queries.

I use the MySQL slow query log to find out which queries were taking the most time -- that's been a wonderful tool that I plan to blog about later.  I will note that the output is much easier to make sense of if you parse it first with this script.  (It's really old, so I had to modify it to look at Query_time instead of Time)

Using the EXPLAIN command for each of the slow queries identified in the above log, I found one that was especially disturbing.   According to EXPLAIN's "extra" column, MySQL was resolving these queries with two fairly expensive operations: 

Using where; Using temporary; Using filesort

Fixing the "Using temporary" required some nimble manipulation of indices, but filesort was really perplexing me.  I wasn't using an ORDER BY clause, so as I read the docs, there was no reason to be doing a filesort.  But then I remembered that MySQL automatically does ordering based on GROUP BY clauses.  All I had to do was add "ORDER BY null" to the end of my query, and that did the trick:

Using where; Using temporary

If only all optimizations were so simple.

Looking for Rails developers (who isn't?)

For the past seven months I've been building a cool new consumer web app with some folks in Austin, TX, otherinbox.com/, and we're looking for experienced Ruby On Rails developers. We're a small agile team led by Steven Smith, founder of FiveRuns.

The whole site is Rails-based and makes extensive use of Amazon Web Services (EC2, S3, and SQS) and there's a new awesome challenge every day. More info on our jobs page.