Building the Tumblr Neue Post Format
Weâve been looking at improving the posting and reblogging experience in our mobile apps for a long time. As many of our power users and public API consumers are aware, posts on Tumblr are stored and served in a sanitized HTML format. This choice made the most sense when Tumblr was originally built, when using Tumblr meant visiting via a web browser on your computer on the information superhighway back in 2007.
Storing post content primarily as HTML has remained our standard for ten years; there are a significant number of assumptions in our codebase about posts being primarily HTML. To compound this, when we want to change something about how posts are made or stored, we have to think in terms of the 150 billion posts on Tumblr and the billion new posts made every month. We have to spend a lot of time thinking about that scale whenever we consider how to make posting on Tumblr a better experience.
Over a year ago, Tumblr Engineering came up with a very ambitious idea: ditch HTML entirely and move to a brand new format for how posts are created and stored. HTML is fine, but its scope is limited as it was intended for the browser, long before the concept of mobile apps existed. Conversely, the JSON standard has been heavily favored by APIs and mobile development for years, and feels much cleaner and more flexible than HTML. We can apply an extensible format and schema with JSON easier than we can with HTML.
With this in mind, weâve chosen to write a brand new JSON-based specification for post content. Weâre calling it the Tumblr Neue Post Format, or simply NPF. The NPF specification breaks apart post content into discreet content blocks, each defined by a type field. All of our existing post content easily fits into this kind of specification, affording backwards-compatibility with the billions of posts on Tumblr.
For example, right now when you add text to a post, we store and serve:
<p>Some text in a post!<p>
With NPF, the same thing is created and served this way:
{ "type": "text", "text": "Some text in a post!" }
Those two representations are fully interchangeable, but we begin to gain advantages with JSON for things HTML cannot do well, providing flexibility and extensibility for future integrations. The power of NPF really becomes critical when we want to build content blocks for Tumblr that cannot be easily represented with HTML, such as a physical location:
{ "type": "location", "latitude": 40.739485, "longitude": -73.988402, "map_style": "quirky" }
This new JSON specification also gives us the benefit of not having to worry as much about potential security risks in malicious HTML payloads in post content. Moving from HTML to JSON allows us to have safer, more injection-proof defaults, and prevents us from having to do heavy DOM parsing at runtime, which means improved performance of our backend and mobile apps. With NPF, posting and viewing posts on Tumblr should be considerably faster and safer.
Our work so far with the NPF specification has been to reach feature parity with the rich text editor available to Tumblr users on the web, as well as extend those basic options with new ones, such as fun new text styles:
{ "type": "text", "text": "Oh, worm?", "subtype": "quirky" }
Our initial release includes support for text blocks (with inline formatting), GIF search blocks, and image upload blocks. All of these options are available in our mobile apps via the Text, Quote, and Chat post forms, as well as when you reblog a post. Yes, you can now upload images in a reblog on mobile.
Future releases of the mobile apps will continue to close the gap with our other post options as we build NPF support for link blocks, video upload blocks, third-party video and audio blocks, and more. We also plan on allowing third-party API consumers to view and create posts using the new specification sometime in the future.
- me (@cyle) and @noisysocks (with love for @keithmcknight who started the original NPF spec)
