#hateos

Last night, I read this post by Matt McClure. It started me off on a frantic bout of clicking, tapping and reading. I'd never heard the term 'hypermedia' defined before, although I knew the basic idea behind HATEOAS - returning information about application state along with entities requested from an API.

For some reason, part of this concept stuck in my head. I've been chewing on it overnight, and I thought I'd distill it into a post before I get distracted by the next shiny thing.

I apologise in advance: this is ill-researched, quickly written and a little gushing. However, I am excited. Read on.

Semantics

A lot of people have heard about 'The Semantic Web', but very few of us have genuinely benefitted from such a thing. The basic idea is to write the html pages that make up the Web in a way that can be understood by machines.

This assumes a couple of things:

1. The average web author is going to have the tools, time and inclination to grok some rather complex concepts and incorporate them into their pages.

2. It makes any sense whatsoever to expect machine-readability and human-readability to coexist in the same data.

There are certainly arguments in either direction, but for where we are at right now, neither of these assumptions make much sense to me. The Web has been a wonderful force for democratisation; you cannot expect the general population to write their html in a semantic way. While simple website authoring tools are appearing, they still aren't at the level that will make this stuff automatic.

More importantly, I think the merits of merging human and computer data into the same documents are dubious at best. A company developing an application has separate processes for user interface, back-end and API design. Attempting to tie all of these together increases the complexity and difficulty of developing any given feature.

Suppose I want to rapidly iterate my UX to optimise for sales or engagement. What if I don't have the resources to also iterate on semantic content? Should I expect every designer and front-end engineer to also be a semantic expert? Not in 2012, buddy.

Simplicity

The idea behind the semantic web is a noble one - make information computer-readable. It's easy to describe, but complex to implement. It has been on the way for decades, and it seems to me that we've bitten off more than we can chew. The standards that are creeping glacially forward don't offer enough immediate benefit for the average web developer to pay attention.

I'm thinking about something simpler.

Hypermedia APIs are those that can describe themselves. That's great, if you're a developer, since you can read API responses instead of documentation. Requesting a 'customer' resource gives you the urls for the 'orders' resource. If you're lucky, there might even be some explanation of what to expect when you GET there.

This doesn't bring us any greater depth of functionality, though. It's convenient, but implementing an API client is still the same tedious process.

What we need is some of that machine-readable magic. We need our computer to know what an 'order' will look like before we request it.

Interfaces

If you're a web developer, you've probably met OpenGraph. Practically, it's the way you describe your website so that Facebook understands what the content means. Conceptually, it's a set of object types - 'article', 'website', 'product', etc. It defines the basic attributes those objects must have ('title', 'author', etc).

What we need is OpenGraph for EVERYTHING. There is some commonality between every 'product' in every e-commerce API in the world. There are shared attributes between every single 'user'. We need to define and describe every common web entity - 'order', 'article', 'message', 'check-in', etc - so that anyone implementing an API can use the same data structure with the same basic attributes (as well as any details specific to that particular service).

Then, your HATEOAS response can contain not just the url of the connected resources, but their type.

This is important because it's what makes our API go beyond 'machine-readable' into being 'machine-comprehensible'. Once your client actually understands the content it is receiving, amazing things become possible.

Dependencies

Google want to organise the world's information. That's a big task, and one I'd rather not have to do. In my ideal world, the information would organize itself. That's probably easier than you'd think.

Whatever modern web language you use, it probably has some fancy dependency management. There's likely a central repository of libraries - like ruby's gems, or node's npm. The repository describes what each library does and the dependencies that it has. Library authors are responsible for uploading and maintaining their code.

Imagine having this for APIs. There's a description of what it does - hopefully placing it into a coordinated 'OpenGraph'-esque 'type'. There's description of the 'types' of resources it can both accept and return. This would be useful anyway, but the thing that takes it beyond our current capabilities is computer readability.

As an online shopper, I want the cheapest copy of any given book. I don't care where it comes from, and I don't want to spend 20 minutes exploring different websites and apps with different interfaces to find it. Google have tried to parse human-readable websites into computer-understandable data, but each new website requires a manual step for it to be available to the consumer.

The Universal Client

If we had a central list of APIs and their capabilities, my shopping client could just query the repository for a list of all APIs that could return 'product's of type 'book'. If I've launched a new book store, I can just describe my API, add it to the list and instantly start appearing in customer's search results. Money will pour in. No manual steps involved.

Extrapolating from this, I want to be able to ask Siri to find the cheapest flights to London, to buy me a book to read on the way and find me a scarf to wear when I get there, then post to Facebook announcing my trip. Siri just needs to understand the words 'book', 'scarf', 'flight, 'London' and 'Facebook'. It can then reach out to the API repo, ask who can supply those resources, request some, then send my credit card information. I could even say 'fly me to the city where my brother lives' and Siri would know what to do. A developer has never had to think about writing that combination of requests; the customer basically just wrote their own program by speaking.

THIS IS THE FUTURE. IT WILL COST MOST OF OUR JOBS AND BE TRULY AWESOME.

Being realistic

As our online data becomes more structured, we're going to get to do this cool stuff. It isn't going to be immediate, and there'll be a long adolescence where some services support this kind of communication and others don't.

We need to build something loose, lightweight and flexible that can grow alongside the industries it is attempting to support. Like OpenGraph, and like Gems or NPM, we need to build something that provides developers with genuine benefits and increases their control over the systems they are building.

Rather than get everything perfect before we start, like the Semantic Web has tried, we can make it a bit crappy for the sake of speed. We can hack together existing technologies, make something that barely works and prove the point that it is extremely cool and will make everybody rich. At that point, clever people will start getting interested and the big problems will begin to take care of themselves.

The first step would be to build a simple set of object definitions. OpenGraph has basically already done this. We can build on it, adding fields for unique object identity (a particular shoe from Zappos) and types that describe API capability ("I can return a list of purchasable items in the category 'pet food'").

The second step is to build a repo for APIs and descriptions of what they are capable of. I'm thinking about doing that now (it would take ~20 minutes to implement naievely).

The third step is to write some sweet APIs, or wrap some existing ones to show what is possible. This is the part that needs doing.