Opperator Blog

As you'll see on the top of our blog, on our Twitter account, on our homepage, and everywhere else we might have a presence, Opperator has a new face! Meet Oppenheimer, the friendly Opperator robot:

As a company our mission is to make things easy for developers. That's it. We wanted any images associated with Opperator to say "friendly" and "developer"...what could be more developer friendly than a robot? Maybe an eight-legged feline, but that's about it.

Branding is important (it's how you set the tone for your company before people have actually learned anything about it) and we were fortunate enough to have friends and colleagues that put something together that we can be proud of. Below is an awesome early sketch of Oppenheimer before he was professionally vectorized into the state you see him today:

If I told you that a page took 130 requests to load 170 KB of data? Would you call that a slow page?

This is a trick question, and the numbers I gave are a double red herring. A page is slow when it feels slow to a user. Without having opened up the page for yourself in a browser, you can't have a real answer.

But if your gut reaction was "yeah, that's damn slow", I wouldn't blame ya. It sounds like a ton of requests for the amount of data being transferred. To disprove that the page is slow, here's a screenshot of the requests in Chrome:

The page is question is the index page to my Facebook profile. Notice how almost all of the requests are being sent in parallel, and most of them finishing concurrently before the actual page skeleton has finished receiving the data.

As web developers, we've internalized two guidelines:

minimize the number of requests

reduce the size of the response

By themselves, these are good guidelines for web throughput performance. Where we go wrong is when these guidelines are applied without considering how the user will perceive the page load. There are times where it makes sense to have more requests rather than fewer in order to reduce page load times. What we should prioritize is page latency rather than page throughput. Facebook wrote a great blog post on this subject. They decomposed a large monolithic page into a bunch of independent 'pagelets' which can be generated, fetched, and rendered independently of one another.

With Opperator, we recognize speed as a feature. And by designing our architecture to be a set of loosely coupled services from day one, it was natural and easy to make our pages feel fast. On top of the advice listed by Facebook, here are some tips and notes about how we keep our pages speedy.

Serve What You Need

Don't serve a giant JSON object of all a resource with all of it's associations. Instead, serve only what you need, when you need it. For example, in our models, a Project has many Services. But when you make a request for a Project, the JSON response only includes a list of Service id's rather than the full JSON for the Project. Of course, sometimes it makes sense to pre-fetch associated objects, but this shouldn't be the default case.

Serve and Render Visible Things First

As users, we don't feel page throughput, we feel page latency. It's ok for your page to take slightly longer time to load, as long as everything the user cares about can be seen as quickly and as smoothly as possible. The priority we choose to serve our content is:

CSS goes first to make sure the rendering of the page feels smooth and natural. We don't want any flashes of unstyled elements.

HTML page skeleton is loaded next to give overall structure and flat content to the site. Flat content anything that isn't session dependent or javascript dependent. Having this structure static will also help your SEO rankings because no dynamic content rendering needs to happen server side. This topic deserves it's own separate blog post.

Javascript is compressed loaded with require.js asynchronously with web modules. Javascript builds all the interactive elements on the page.

Images are sprited and loaded as late as possible, with the exception of anything that makes the page reflow and jerk.

Render Client-Side

Our business logic lives within a Grape API. The initial page load is a static HTML skeleton that is then bound to Javascript event handlers which will populate the content from the API backend. We use Ember.js (aka Sproutcore 2.0) to control and render all the frontend interaction.

Skip Duplicate Work

Set your HTTP cache headers properly. If nothing has changed, then your backend doesn't have to regenerate a full response. Better yet, if something can safely be cached for longer, then your backend might not be hit at all.

CDN Your Static Assets

Think of a CDN as a proxy layer in front of your app. By offloading your static assets into the CDN, your app no longer has to worry about serving those static assets. Not only is it less work and bandwidth on your backend, it's also faster for your users since the CDN's edge servers will be physically closer to your users. Two low-cost CDN providers worth checking out are:

Cloudflare is a relative newcomer to the arena. They range from free to $20/mo for your first site. That's hard to beat.

If you store your static assets in S3, then Amazon Cloudfront is worth a look. It allows you to configure serving S3 assets through their CDN. Pricing is based on usage.

Defining "Fast Enough"

Our website is actually deployed to a different server than our API. Technically, we could get a performance boost if they lived on the same box to cut down on network latency. But we decided to tradeoff network latency for clean separation of systems. There's always room down the line for further optimization, so "fast enough" for today is good enough for us.

When building a product for your audience, you need to empathize with their pains and problems. It's one thing to imagine yourself in their proverbial shoes, but it's much more effective to just wear the same damn shoes. It means using your own product while you're building your product. Startup slang commonly refers to this as "dog-fooding", but I dislike this term because it evokes images of being forced fed something you don't want.

While it's important to focus on relieving your customers' pain, it's equally important to optimize your own happiness along the way. Not only will you enjoy your work more, your code will be "happier" as well. Allow me to explain what happy code looks like.

When Michael and I started Opperator, we aligned ourselves with our customers. When life sucks for our customers, life sucks for us. When life is awesome for our customers, life is also awesome for us. It's a very simple formula to balance: remove sucky things, and increase awesome things. Simple as this sounds, many entrepreneurs only tackle the former half of the formula. For any startup, there are many things that happen behind the scenes that your customers don't see. Things like code smells, technical debt, and shoddy documentation. But just because your customers don't see these things, doesn't mean that you don't.

We execute quickly on Opperator, but we also take the time to step back and holistically evaluate our product.

Are we happy with the direction we're going?

Are we happy with the code we've written?

Do we feel mentally or physically burned out?

So a month or two after writing a blog post about keeping blog content flowing ours has slowed down a bit. Why? We're building our product and despite our best efforts neither of us has been able to dedicate time to fleshing out our written content.

Building an API driven Javascript app brings about challenges for authentication that I've taken for granted when working with traditional http frameworks. In this post, I'll outline the basic concepts of what a session is, and how we manage sessions with Backbone.js.

HTTP is a stateless protocol. This means that nothing is remembered from one request to the next. This is great for normal content sites, but lacking when building a webapp. A normal HTTP request lifecycle looks like this:

But for webapps, we need to have a concept of a logged in user and unauthenticated user. Web frameworks hide this in a Session abstraction. The gist of how an authentication request flow works look like:

First a user will submit a login form to the backend. Instead of just returning a response, the backend will generate and remember a session id that it can use to look up this particular login for later requests. This session id is passed back to the client as a cookie value.

Now whenever the client makes a request, the cookie will also be sent to the backend. The backend can then check to value to know whether the user is authenticated and authorized to access the resource they're requesting.

With the theory part out of the way, here's how we structured authentication in Backbone.js. First, we created a Session model.

App.Models.Session = Backbone.Model.extend defaults: access_token: null, user_id: null initialize: -> @load() authenticated: -> Boolean(@get("access_token")) # Saves session information to cookie save: (auth_hash)-> $.cookie('user_id', auth_hash.id) $.cookie('access_token', auth_hash.access_token) # Loads session information from cookie load: -> @set user_id: $.cookie('user_id') access_token: $.cookie('access_token')

This model has a simple and straightfoward interface. When it's initialized, it looks for an existing login with the jQuery cookie plugin. If it finds this information, it'll set the access_token and user_id for later use. Once an access_token is set, then the authenticated method will return true.

In our main application, we'll have a single entry point for our application that'll route the user the based on whether they're authenticated or not.

App.start = -> @session = new App.Models.Session() if @session.authenticated() # redirect to user page else # launch a login form

With this, all of your pages can run App.start() at the bottom of a page to initialize any authentication logic.

There is one ability that I've come to learn can be a pretty good indicator of the experience level of a programmer; an ability that I almost always see experienced developers do well but that I've rarely seen novice to intermediate developers do well if at all. I'm talking about being able to debug code fearlessly.

I feel confident that, given enough time, I can debug (nearly) any problem in any code in any language (whether or not I'm familiar with it). This is primarily what I mean by fearless debugging. It's an ability that I think grows over time as you become confident in your own grasp of programming's underlying fundamentals. Note that this is not the same as being fast at debugging or being familiar with great debugging tools. It's about being able and willing to follow a backtrace all the way into the depths of hell if that's where it takes you and come out with a solution on the other side.

A key differentiator in debugging ability is where one draws a line of "beyond my understanding". Here are some rough categories for this:

Brand New Programmers: The error message. They may not have enough context or confidence to understand the error message and act on it, so they immediately go to Google and then to asking for help if Google fails.

Novice Programmers: Code that they wrote themselves. They can understand the error message and follow it back through their code, but if they didn't write it they fear not being able to understand it and so ask for help.

Intermediate Programmers: Code in their application/domain. Willing to follow code back to whatever part of the application is causing the problem and fix it there. However, if the problem lives in library code, they will probably stop there and ask for help.

Advanced Programmers: What line? Willing to pull up the source of libraries, debug the problem there, and maybe even submit a pull request for the fix afterwards. There are different levels within this...I'm willing to follow problems pretty far but not all the way into language errors. Those who are able to go all the way to the metal (the source code of Ruby, for instance) have a tip of my hat for their skills and determination.

Having come to this conclusion of a potential differentiator, I wonder if a useful interview technique could be to introduce an intentional (perhaps somewhat subtle) bug into your codebase and watch as the person debugs it in front of you. Debugging requires a lot of quick thinking and on-your-feet problem solving that may be a better window into the potential hire's development process than asking him/her to slap together some contrived example, especially on a whiteboard.

We're taking a break from our normal blog schedule this week because, well, it's the holidays and no one's reading anything anyway. So whether you're taking time off to hack on personal projects or just spend time with family, we hope it all works out for the best.

It's time to come clean and admit that I have a problem. I find myself irresistibly drawn to new development technologies, able to immediately grasp how much better my development life would be if I were using these technologies. My name is Michael, and I'm a neoholic.

The good news is that being a neoholic isn't usually a bad thing. It fuels my passion for development and keeps me abreast of interesting new ways to do things. If I weren't a neoholic I would have never bothered to learn Ruby, never would have found out how much better things are with SASS and CoffeeScript. I might never have tried Backbone.js and shattered my perception of the responsibilities of the server and client sides of apps. I certainly wouldn't have built Grape or OmniAuth if my drive for newness didn't lead me to forge my own path when I could find no way ahead.

Seeking newer and better tools to do your job more effectively is a positive trait, but there is a dangerous flip side: seeking newer and better tools for their own sake, without any real gain in practical development. Let me explain by example.

We're building a marketplace for developer services. Building a developer-focused startup, we knew up front that we should work to architect the application in a way that would help us easily release a first-class public API. So our core application is actually the API itself, the web interface was just to be a simple client that would proxy through to the API.

"A ha!" I said. "Since we're basically just building a client-side JS app, we should write the server-side of the web client in Node.js. Javascript everywhere!" This remains a perfectly reasonable conclusion. What I wasn't considering, however, is what I'm calling the ecosystem cost of two primarily Ruby developers deciding to build something (ostensibly simple) in Node.

This is not a disparagement aimed at Node, by the way. Node.js is an awesome technology and I've participated in both years of the Node Knockout. I still believe that it may be the perfect-world ideal solution for the server side of our web client. What it wasn't was the real-world ideal solution.

While scrambling to learn a whole bevy of new tools to get things up and running the way we wanted (such as node-http-proxy for API proxying, RequireJS for asset loading) and simultaneously trying to use a literally just-released-and-still-sort-of-unreleased framework on the client side (Ember.js), we were getting very little done towards actually building our web client. While the technologies were (and still are) all great ways to get things done, the problem was that neither of us was familiar enough with them for us to get things done!

A short while ago we said ENOUGH! and talked about switching back to Ruby for the server side. After about an hour and a half of work I had a simple Sinatra app using rack-proxy to proxy to the API, Sprockets to package assets and compile Coffeescripts, SASS, and Handlebars templates, and jasmine-headless-webkit running in conjunction with Guard for automated, headless full-stack javascript testing of our client-side code. Just like that, I felt productive again.

What did I learn from all this? New technologies, like all things, are best used in moderation. Now, my definition of moderation is pretty liberal. If I was really going back to my comfort zone I would have just run back to Rails when Node.js wasn't moving fast enough, but that didn't feel right for such a lightweight client. Rather than learning an entire stack of new technologies, however, I simply composed technologies I already knew in a novel way that best suited my needs. We're still exploring Ember.js for client-side code (I'm not entirely sold, but the bindings are really nice) but we're building with Backbone for now because, again, we can Get Things DoneTM.

The other thing I learned from all of this is gaining a lot more sympathy for those developers who are using familiar technologies rather than striking out with something new. It's easy for neoholic developers to turn their noses up at PHP or .NET developers, but keep in mind how big of an investment learning a new technology can be.

MicroGems are small reusable libraries not quite big enough for their own RubyGem that are hosted as a Gist on GitHub. As we create these for Opperator, we blog about them!

In Opperator we're in a position to need a few "random token" attributes. You know the kind I'm talking about, a randomly generated token that is used for security-through-obscurity. Could be used for a private-but-public URL slug, for an API token, or lots of other things.

Installation

In your application's Gemfile:

gem 'mongomapper-tokenkeys', :git => 'git://gist.github.com/1428845.git'

Usage

I wrote a small, simple plugin for MongoMapper that allows you to declare these fields in one line. They are then generated before validation on model creation. Use it like so:

require 'mongomapper-tokenkeys' class User include MongoMapper::Document token :api_token, :size => 30 end

The generated tokens use SecureRandom.urlsafe_base64 so they have a large character set but can still be used as URL parameters. The :size that you pass doesn't map directly to the character length (a quirk of SecureRandom) but a bigger size is a longer token. If you don't specify a size, it will default to 20.

While you're building your application, chances are you'll run into more than a few times where you write some kind of library-esque code that you then, of course, drop into a lib folder somewhere. Sure, it's pretty reusable and you may want it in another project someday, but it's just one file, surely it's not worth the headache of releasing it as open source, right?

Wrong. Every piece of library code living inside your application is a piece of code that you are solely responsible for maintaining. Even if it's just one file it can often be worth it to publish it as open source just to keep it out of your application.

If you're using Ruby and Bundler, a dead simple solution to this problem is to create MicroGems. You can store your library code in a gist, write tests, and package it as a gem. Now you can use it via git in your app and your library code is no longer a concern of your application.

So what are the benefits of libifying your application? Here's a few that we're noticing already with Opperator:

Simple App Code. Dedicating ourselves to writing as many libraries and as little app code is making most of our business logic implementation trivial. We abstract out the tough stuff and reading our app code becomes easier and easier.

Contribution. If you release your app code to the community, that gives the community the chance to improve upon it. Maybe for MicroGems or other tiny libraries you won't see the contribution benefits of a full open source project, but it still leaves the opportunity for members of the community to reach out and help you.

Goodwill. The developer community appreciates companies that go out of their way to try to give back. You will build a better reputation with other developers by releasing code. For us, this is a direct benefit since developers will be our customers, but for all companies it can help in a myriad of ways including recruitment and reputation.

Modular Thinking. You will build a better application architecture if you try to make nearly all of your library code package-able as open source. If you think of each piece of library code as a standalone module that needs to be able to easily plug into your app and get the job done, you will build it that way instead of having tightly coupled dependencies between pieces of your app.

So why am I advocating MicroGems instead of full GitHub repos and RubyGems publishing? Because for all of the great things open source gives you, it also saddles you with responsibility. If your open source library is hosted as a gist, people will be able to instantly understand that this is a small project and may not have someone dedicated to its independent maintenance. And if your MicroGem grows and you (or someone else) think that it deserves its own dedicated repo, that can happen organically and at a time that makes sense.

As developers, we want the world to behave in a rational way. Part of this, I think, is a desire for the world to be a meritocracy: if you create value, you receive compensation proportional to the value you create. This applies in plenty of spaces (such as salary), but I think it especially applies when it comes to building a product. As a developer, I don’t want to have to think about advertising, marketing, sales, bizdev. I just want to build something cool and then have people start using it.

Start a company? Yech. Most hackers would rather just have ideas. But as Larry and Sergey found, there’s not much of a market for ideas. No one trusts an idea till you embody it in a product and use that to grow a user base. Then they’ll pay big time.

Paul Graham — “The 18 Mistakes That Kill Startups”

Execution of an idea is extremely important, that’s become a part of the entrepreneurial common knowledge set. But what can be frustrating (it is for me) is that even if you execute well on your core product, you still have little guarantee of success. External factors like marketing, audience building, and cost of engagement can all affect your product’s ultimate success just as much as your execution.

Some might argue that dealing with these factors is part of execution, and that’s a fair point. But does it have to be that way? Does the success of a product or service have to be tied to things that are historically in areas outside of developer expertise? The simple fact is that I want to be able to launch a business just by building something cool. I don’t want to manage all of those external details, I just want to ship something cool.

Enter App Stores

When Apple launched its App Store a maelstrom of activity developed around building apps for iOS. Why did that happen? Because Apple gave developers an out for the external details. To have a successful app on the App Store (at least early on, now the dynamics have changed as the market has become crowded) you didn’t have to market yourself, you didn’t have to know how to reach out to an audience. You just had to build something cool and people would find it and use it.

Developers flock to app stores of all kinds because they abstract away so many of the implementation details that go beyond just building a good product. You don’t have to worry about payment systems, you don’t have to worry about distribution, you don’t even have to worry about creating some kind of landing page to promote the features of your app. You just build it, release it, and (hopefully) profit.

The powerful draw of these abstractions is obvious when you consider what developers have been willing to put up with in Apple land: arbitrary and constantly changing rules about what can and can’t be a part of your application, an approval and release process that is entirely outside of developer control. The convenience of a central distribution channel is so powerful that developers are even willing to give up 30% of their revenue to be able to leverage it.

Platform as a Service

Of course, it isn’t only marketing and distribution that can frustrate the “just build it” mindset of a developer. More and more, developers also don’t want to have to worry about their operations infrastructure (at least not until they need to worry about it because of scale). Platforms like Heroku abstract away that infrastructure, giving developers still one less thing to worry about.

The simple power of being able to provision and run a public application server for my web app in a matter of two or three minutes means that I deploy my projects early and often and will never run into launch delays because of operations complications. My production environment is live from day one and I can just focus on building a good product.

We’re Getting There

In conjunction, App Stores and Platform as a Service providers can get you much closer than ever before to that theoretical meritocracy where products are used based on their relative usefulness and interest rather than because they had all of those external implementation details just right. Which isn’t to say that those external implementation details aren’t important: they are, I just don’t want to have to deal with them myself.

Behind the Curtain posts explore the development of a piece of Opperator's architecture, including the what, why, and how.

Grape is a Ruby framework for building restful APIs on the web. We're using it extensively to build Opperator. Out of the box, Grape has built-in support for versioning your APIs. There's general information about how to use versioning in the Grape README and the wiki, but this post is more about digging into the nitty-gritty and discussing some of the design decisions and implementation details that've been merged in recently.

Path Based Versioning

Previously, Grape supported API versioning by prefixing the version name in the url. For example:

class MyAPI < Grape::API version :v1 # GET /v1/cows get '/cows' do # retrieve bovine goodness end end

When version is called, the version names are passed into Grape::Middleware::Versioner pre-refactored file When a request comes in, the middleware looks for a matching version in the path. If it finds the version, it rewrites the path info without the version prefix, sets env[api.version], and moves along it's merry way. If no version is matched, then a 404 is thrown.

Header Based Versioning

This works as you'd expect, but introduces your versioning scheme into your resource uri's. Workable, but it messes up those pretty restful uris. Fortunately, the HTTP protocol Accept header is a perfect fit for this problem. RFC 2616 defines the Accept header as:

The Accept request-header field can be used to specify certain media types which are acceptable for the response. Accept headers can be used to indicate that the request is specifically limited to a small set of desired types, as in the case of a request for an in-line image.

While the example the RFC gives is related to multimedia, if you squint and replace the references to media with 'version', then you have a good overview of header based API versioning. This Accept header field can be used to scope a request to a specific API version. For example, the Github API understands the following Accept header:

Accept: application/vnd.github-v1+json

The client who sent this header is asking the server "Hey Github, can you give me a responses that is version v1 and formatted in JSON?". When Github sees this request, it can do one of two things. If it's able to answer the question, then it processes the request as normal. However, if Github doesn't understand the Accept field value, then it should send a 406 Not Acceptable response.

Revisiting our code sample, we would define our API as follows:

class MyAPI < Grape::API version :v1, :using => :header, :vendor => 'intridea', :format => :json # GET /cows get '/cows' do # retrieve bovine goodness end end

Header based versioning is the new default versioning strategy, but I explicitly specified it in the example for clarity. The vendor option is new and is a way to describe the vendor providing this API, and the format option is the expected response format. Similar to path based versioning, Grape::Middleware is responsible for figuring out the version being requested and setting env[api.version]. But since there are now multiple strategies for handling versions, Grape::Middleware::Versioner has been split into two middlewares. Grape::Middleware::Versioner::Path is the original path based middleware, and Grape::Middleware::Versioner::Header is the new kid on the block. Relevant commit here

This new middleware will use the following format in the Accept header when matching for versions:

application/vnd.:vendor-:version+:format

These are the fields that are original declared when version was first called. If the middleware is able to match these fields, then the endpoint is called with some extra environment variables 'api.vendor', 'api.version', and 'api.format'. If the version couldn't be matched, then the middleware returns 404 and also sets the X-CASCADE header to pass. That last part is important because it allows Rack::Mount to keep looking for other endpoints which might match the version.

You can also control the routing behavior when no Accept header is specified with the strict option. If strict is set to true, then a 404 will be returned when no Accept is set. If strict is false, then the first matched endpoint is returned. This is inline with the RFC definition and basically means that the client doesn't care which version the server responds with. Most likely, if strict is set to false, you'd like to use the latest available version. To achieve this, you should mount your latest version as high as possible (similar to routes precedence in Rails)

class MyAPI < Grape::API # version v2 has higher precedence than v1 version :v2, :strict => false do get '/cows' do end end version :v1 do get '/cows' do end end end

Custom Versioning Strategies

Currently, path and header based versioning are what's understood, but we've opened up the possibility of custom versioning strategies. Prefer to do domain based versioning? Or IP-based versioning? If you want to get really wacky, you can even version based on the lunar calendar.

This may be reading too much into random and unrelated things, but the idea of charging people for access to web APIs seems to be coming up more and more around the web. I saw it the other day on a Forbes article about Facebook (saying that one possible revenue stream for FB is to charge for Facebook Connect) and then in a random developer's post about API-driven applications (saying that monetizing APIs is an good reason to build API-centric applications).

Side Note: I have no beef with the latter post in general...it was mentioned in just one paragraph in his post and he's encouraging people to do something I wholeheartedly support (building your API as the core of your app). This just brought up an issue that I have strong opinions about and wanted to take some time to cover.

So to everyone out there who thinks it might be a good idea to charge for your API, I have one question for you:

There are a multitude of problems with this attitude. I'm going to tackle one on a specific front and then some more in a general manner.

Charging for Facebook Connect: A Terrible, Horrible, No Good Very Bad Idea

Forbes thinks so highly of charging for access to Facebook Connect that they listed it as one of only three potential ways for the social networking behemoth to make money. Wow. Maybe I should forgive them, Forbes is a financial publication, not a tech one, but just wow.

If Facebook started charging for its development platform, that would do exactly one thing: cripple its development platform. From my experience Facebook already has a tenuous relationship with developers (built more on a sense of feeling obligated to integrate with Facebook than a real joy in it) and adding a toll booth at the entrance to Facebook integration is only going to snap that tenuous link.

That's not to say that some wouldn't pay the price. All of those multi-billion dollar brands would probably happily still pay for their Facebook promotions. Zynga would likely pay up (begrudgingly) so as not to lose their stranglehold over the free time of the casual Facebook gamer. But smaller companies, individual developers and startups? No thanks, I can build on top of someone else's social graph if you're going to make me pay for Facebook's. I hear Twitter is integrated into the iPhone these days.

Charging for Facebook Connect (which is not even what it's called anymore) is also completely orthogonal to Facebook's company mission to Borg-style assimilate the concept of identity on the web. It is to Facebook's enormous advantage to get every site they possibly can integrating with Facebook, because that only drives more data and more eyeballs to the advertising and other means that will actually make Facebook money over time.

OK, so now I've got this little rant about Facebook off my chest, but let's talk about the general question because I can see how it would be tempting to charge developers for API access as an additional revenue stream for your startup.

Why You Shouldn't Charge for an API

If you're building a startup (especially a consumer facing one), monetization might be a tricky question. You may have pressures from investors or even just yourself that say "I need to know now how I'm going to make money." In general, that's a good thing, I think the "just get an audience and money will work itself out thing" is a dangerous line to tread. However, if you're thinking monetization and "I know! We'll charge developers to access our API!" I'm here to put a stop to that madness before it gets worse.

API friction is already a problem. It's hard for developers to integrate with new APIs. They have to learn a whole new set of methods and no two APIs are really implemented alike. You will likely already have to pour blood sweat and tears into docs and developer relations just to get a decent community using your free API. Charging for it is only going to increase this friction.

Your API is not your business. You are presumably creating a product that has some sort of business value beyond its API (if your API actually is your business, maybe you can charge; see the section below). But the moment you start charging for access, your API is your business. And paying customers are going to have big expectations, meaning you now have to serve your API customers in addition to your normal customers.

APIs should enhance your core business. Do you remember why you wanted to build APIs in the first place? It was probably in the hopes that developers would make awesome tools for your product that you didn't have to build but nonetheless drive additional audience and/or customers your way. Charging for your API makes it more likely that developers are going to charge for their API products and you lose out on this benefit almost entirely.

Developers will go elsewhere. The problem with marketing yourself towards developers (and this is what you're doing if you charge for API access) is that they're going to be a savvy bunch. If they can find a way to do for free what you offer for charge, they're going to flock to it. Expect zero traction unless your product is so unique and peerless that they have no choice but to buy in. And even then, many developers would just take their toys and go home rather than integrate with a paid API platform.

Hopefully this gives you some idea of why charging for an API is generally a massively bad idea. Of course, there are exceptions to every rule, and I'd be remiss not to point out that sometimes charging for an API can make a whole lot of sense.

When is Charging for an API OK?

So when is charging for an API OK? When your API is your product. That's it, that's the only time.

What do I mean by that? Startups that cater to developers by providing means to accomplish difficult-to-do thing via a simple API. Think of a simple API for grabbing screenshots of web pages, or semantic text analysis as an API, or an API for sending and/or receiving faxes. You can charge for an API when your API is going to take pain away from developers because in that scenario the developer friction works in your favor. If your API is much, much easier to use than the free alternative, finally you have the leverage you need to charge developers, and they will gladly pay.

The other potential scenario where it makes sense is when the infrastructure costs of providing your API are so significant that you have no other option. Twitter's firehose is an example of this. The sheer volume of data that has to be transmitted to firehose partners demands payment because it requires dedicated operations support and resources just to make sure it doesn't all come crashing down. And that only works because Twitter already had millions of users when they started charging.

With the year coming to a close and the holidays fast approaching, it's a good time to step back and reflect on how lucky we are to be developers in this point in history. Below's a list of things that we're thankful for.

Jerry's List

Webhooks: As a kid, I loved legos. There's something magical about taking a bunch of colorful rectangular blocks and turning it into something recognizable. Software development is like legos, but with more block types. And as a web developer, web hooks are our version of lego expansion packs. You can build your own expansion and share it with the world. Oh, and on top of that, when you finish, you end up with something useful. How cool is that?

Emacs: I first used emacs in my first computer science class at Cal (CS61A Go Bears!). While most of my friends are Vim snobs, Emacs will always have a special place in my toolbox. The endless customization is addictive and the macro capabilities can't be beat. The UX/UI leaves something to be desired, and someday I hope to write the beautiful UI wrapper that it deserves.

Ruby: This one's obvious. Whether I'm using it in Rails for web development, hacking on Grape for APIs, or batch processing images, Ruby just makes it fun.

Michael's List

GitHub: Do you remember the time when you had to apply and get approved for open source project hosting? Thanks to GitHub those days are behind us. The open source community owes a huge debt to the octocat for increasing the pace and ease of open source development by an order of magnitude or two.

Low Overhead: So far for Opperator we have a beta signup system, a blog that can handle thousands of visitors, a deployment environment for building our app and a place to host our code. Total outlay so far? $10 for the domain. Okay we're piggybacking on the private GitHub account for Intridea but that's still amazing.

Other Developers: The ones that fix stuff on my open source projects, the ones that hash out ideas with me and help me better my game, the ones that spend countless hours building things that I get to use for free. The ones that answer my random questions on Twitter. You guys are all awesome.

Software development is about building cool stuff. The path isn't always perfectly smooth, but it is buckets full of cranberry awesomesauce. So take a moment and reflect, what are you thankful for as a developer?

A lot of startup advice revolves around launching a minimal viable product quickly. You’re happy to put in plenty of sweat and elbow grease for your fledging product, but what about all of your non-core competencies? How will you collect emails from interested users? Or how about keeping your audience engaged on your blog? You want something simple to setup, but with enough functionality to get stuff done.  Without further ado, here’s the list of tools we used to get Opperator’s community machine up and running.

LaunchRock

A simple no-frills service for building a landing page to collect contact information from interested users. We chose it for its built-in analytics and simple styling. Protip: the customizable fields allows html, so you can tweak their default form styling.

Tumblr

We signed up for both Posterous and Tumblr, but ultimately went with Tumblr. Being developers, we really wanted something that supports Markdown. We wanted flexibility and features, but at the same time, we didn’t want to get distracted and setup our own blog system.  Remember that every hour you spend shaving that hairy blog yak is an hour you could spend on engaging users and refining your product. While Tumblr has its quirks (its UX infuriates me), it’s very easy to style and easy to manage. Extra tip: to add images to your text posts, consider snapping screenshots with Cloud App or Skitch.

Twitter

Since we’re building a product for developers, being reachable on Twitter is just as or more important than having an email! We’re on Twitter as @opperatorapp and we’re cross-linking to our Twitter account everywhere. Twitter is such a fast, easy way to engage your audience that it’s a virtual no-brainer.

GitHub

We love open source, and open source loves us back. On top of using GitHub to host our code, we also use its wiki to organize notes, and its issues tracker to manage tasks. Developer friendly APIs makes it a joy to work with, and we’re going to emulate the same developer joy in our product.

Ducksboard

There’s a number of these services out there, but we had a beta invite sitting around for Ducksboard, a simple dashboard for various metrics about your site. We’re still working on wiring it up, but hopefully it will be a good way to measure key data across the business at a glance.

Keeping it all organized

For starters, we're using Highrise to keep notes with people we've talked to, Mailchimp to manage email campaigns, and Google Analytics and Gauges to keep tabs on our web analytics. We'll trim and add services as the need rises.

Tools shouldn’t get in the way.

 At the end of the day, don’t forget that tools are meant to enhance your capabilities as a person. If you find you yourself customizing endless options and seething in frustration, then that shiny tool might really be a liability in disguise.