Dave

Today, we’re abnormally jazzed to announce that we’re open-sourcing the custom framework we built to power your dashboard on Tumblr. We call it StreamBuilder, and we’ve been using it for many years.

First things first. What is open-sourcing? Open sourcing is a decentralized software development model that encourages open collaboration. In more accessible language, it is any program whose source code is made available for use or modification as users or other developers see fit.

What, then, is StreamBuilder? Well, every time you hit your Following feed, or For You, or search results, a blog’s posts, a list of tagged posts, or even check out blog recommendations, you’re using this framework under the hood. If you want to dive into the code, check it out here on GitHub!

StreamBuilder has a lot going on. The primary architecture centers around “streams” of content: whether posts from a blog, a list of blogs you’re following, posts using a specific tag, or posts relating to a search. These are separate kinds of streams, which can be mixed together, filtered based on certain criteria, ranked for relevancy or engagement likelihood, and more.

On your Tumblr dashboard today you can see how there are posts from blogs you follow, mixed with posts from tags you follow, mixed with blog recommendations. Each of those is a separate stream, with its own logic, but sharing this same framework. We inject those recommendations at certain intervals, filter posts based on who you’re blocking, and rank the posts for relevancy if you have “Best stuff first” enabled. Those are all examples of the functionality StreamBuilder affords for us.

So, what’s included in the box?

The full framework library of code that we use today, on Tumblr, to power almost every feed of content you see on the platform.

A YAML syntax for composing streams of content, and how to filter, inject, and rank them.

Abstractions for programmatically composing, filtering, ranking, injecting, and debugging streams.

Abstractions for composing streams together—such as with carousels, for streams-within-streams.

An abstraction for cursor-based pagination for complex stream templates.

Unit tests covering the public interface for the library and most of the underlying code.

What’s still to come

Documentation. We have a lot to migrate from our own internal tools and put in here!

More example stream templates and example implementations of different common streams.