#rspectacles

I don't know about you, but the performance of my test suite seems to be pretty strongly correlated to my own happiness as a developer. When one of my test suites was growing out of hand about a year ago, I decided to take a few days to optimize.

I took a look around at the different profilers, and learned pretty quickly that they were coming up short. The problem was, most profilers focused on identifying the slowest individual specs first. 

It became clear pretty quickly that attacking individual slow specs to shave off a few seconds at a time was not the answer. Most of the slowest specs are slow for a reason - for instance, we test our Credit Card and Order processes end-to-end, which involves setting up a bunch of complicated relationships in the database. I already knew that spec was slow - but the peace of mind it gives is worth it.

I felt that the real win would come by speeding up groups of specs; for instance, identifying slow before, let, and subject blocks that were consumed by dozens of individual specs. In order to decide where best to direct my efforts, I needed better visibility to the problem.

In order to gain this vision, I built RSpectacles. RSpectacles is an RSpec visualizer that streams into the browser in real time, and shows the performance of a test suite using a partition graph:

This powerful visualization allows me to drill down into great detail about the health of my test suite. I went on an optimization spree once the tool was complete, and within a few hours I'd improved performance of the test suite from ~30 minutes to under 10!

An example of one of my big wins was the proper use of the transaction vs truncation strategies for database cleanup. We were using truncation so that our request spec would run properly; but truncation is significantly slower than transactions, and was slowing down the suite as whole.

Once I identified the problem, it was simple enough to use DatabaseCleaner to deal with my strategies. (http://devblog.avdi.org/2012/08/31/configuring-database_cleaner-with-rails-rspec-capybara-and-selenium/) But I don't know of a single other profiler that would have helped identify this problem. I was tipped off when specs that should have taken 5ms were taking 50ms... without RSpectacles there's no way I would have noticed that kind of a delay.