How we used Rails & Backbone.js to build a lightning-fast webapp.
Whenever I build a greenfield webapp, there's a bunch of things that I take into consideration when deciding what tools to use. When we first sat down & started planning out Stride, I immediately started thinking through how I was going to put it together, what tradeoffs we were going to have, and what kinds of tools would fit best.
There were four key things we wanted to solve with our choice of tech stack & architecture:
We wanted it to be fast and responsive to the user. Really fast.
We wanted it to be maintainable moving forward, and make our lives easier rather than harder.
If Stride wound up being a massive success, we wanted to be able to hire against our tech stack. A specific stack or framework affects hiring in two big ways - obviously it's nice when the talent pool for a given skillset is large, but good folks can learn new things easily. What your tool choice does affect, however, is who's interested in working with you. As an extreme example, building something in COBOL will self-select your applicants quite effectively.
This fits into the last one nicely - we wanted to have fun building it.
So with those things in mind, we settled on the following big moving parts:
We're using Backbone.js to provide structure for our frontend, which is a hefty JavaScript app. Everything happens client-side, and is then persisted to the server asynchronously.
The backend is Rails. We chose Rails over other options for a few reasons - Backbone was relatively new territory for us starting out, and if everything went poorly, we could implement more of the app in the Rails side if need be. We thought about working with a lighter Ruby framework like Sinatra or Padrino, but we decided that at least early on, more power & flexibility was better, and we'd optimize later.
Because so much of the app is client-side, we're using a messaging queue built with node.js and socket.io to send server-initiated messages to the client. These might include user-facing alerts (think Rails' flash messages), updates to your dashboard. We're currently using juggernaut, but we're looking at migrating to something else soon. I've had to hack in SSL certificate chain support, and there's a few bugs that we've found, but the big reason we're moving away is we want persistence for our messaging layer. The current frontrunner is spire.io - their team has been very open and forthcoming when discussing our needs with them.
All of our calculations and intelligence happens in a work queue. We're using Resque for this at the moment, but as we move forward and things get more complex, we'll likely wind up with something different at this level. Specifically, as our metrics calculations are going to get a lot more complex, I'm exploring some of the various BigData offerings. I've been exploring Hadoop and Mahout lately, and it's quite likely that we'll be leaning on them. Resque is great for simply getting things asynchronous, but our as intelligence calculations are getting more and more complicated, it's time to move to something more suited for it.
We also have some API endpoints that are built with Sinatra. We hope to expose these soon to developers - if you're interested in building things with Stride, drop me a line - I'd love to chat.
There were some interesting lessons learned, especially with this being our first big backbone.js app as a team.
Having clients be asynchronous was a huge boon for us under high load - the user received feedback immediately, and the actual persistence was able to take some time without the user having to think about it. This worked well for us because only one user can update a given dashboard - once we have multiple users, we'll have to work through concurrency resolution.
We fell into some bad JavaScript practices early on, and didn't build as modular as possible. We expected too much out of Backbone, and didn't do a good enough job of building good, modular code. As with any tool, Backbone's just a tool - it's up to you how you use it.
As you build more complicated user interaction on the client, keeping completely RESTful gets hard. Say, for example when you update Resource A, Resource B might need an update, but it has to be up to the server to decide. If there was a page refresh, that would be fine, but if you're interacting with a RESTful API, how does the server notify the client whether or not to update? Do you always call for an update? Do you attach extra information to the API response? Do you send an out-of-band message back to the client from the server using some sort of messaging layer? We're currently going with the extra information route, but we're planning to move to leveraging our messaging layer in the future.
A framework itself can serve as documentation, but when you move outside that framework, documentation is needed. I don't need to have documentation describing the standard ActionController actions, for example. I don't need to tell someone what "def show" is doing. But once I step outside the framework, it needs documentation. We've got a few places where we're doing totally non-standard things with either Rails or Backbone, and we haven't done a good enough job at explaining them to current or future developers.
So that's a brief overview of the big moving parts of Stride, and some of the lessons we've learned so far. It's been great fun, and I'm very excited with where we're going. I hope you'll join us for the adventure.
If you've got any comments or questions, please drop me a line on Twitter, or join the discussion on Hacker News.
