[Read part 1 of Inside Jidometa]

We recently implemented a feature which greatly improves the browser experience in Jidoteki Meta (Jidometa).

In the past...

The Jidoteki Meta UI is a simple JavaScript layer on top of a PicoLisp REST API. The API returns JSON responses for every successful request, but occasionally we would see rather slow navigation between sections of the UI.

Certain responses tend to take more time as the size of data increases. There is no way around this, since many tables and entries need to be queried on every request, or so we thought..

The present...

To “solve” this problem, we decided caching responses for common GET requests would be a good approach.

Our idea was to only cache the most queried items (the list of all builds, and the details for each build). By caching this data, it makes it possible to quickly navigate between different builds and the full list of builds, without making any calls to the API.

Caching gotchas

Before implementing any caching solution, it’s important to fully understand the gotchas that accompany it. In most cases, the most important requirement is to never serve stale data. To invalidate the cache requires us to ensure any cached elements which are affected by a change, will immediately be evicted from the cache.

Another gotcha, is avoiding the temptation to add yet another layer of complexity to the stack. We always aim to do things as simply as possible with the least amount of software and overhead.

We needed the ability to easily disable caching or wipe the cache if needed, and we needed to ensure the browser’s own caching features didn’t interfere with ours.

Caching as simply as possible

Since our Jidometa virtual appliance (as well as our customer appliances) run entirely in memory, it was easy for us to assign a temporary directory for storing cached responses.

We store responses in /tmp/jidometa/cache/ - which is not actually “on disk”, but rather an in-memory filesystem (tmpfs). Those responses are regenerated on every GET request to the API. Fortunately there’s no overhead for that.

Each cached response is a simple .json file which is absolutely identical to what would have been served by the API. The files are named according to the build ID they represent, so there’s only ever one cached response per build.

It’s hard to get any simpler than that.

Serving the cached responses

We use Nginx as a frontend for static files and the API. Below are examples of the two Nginx directives we added for serving the above cached files:

location /cache/ {  add_header Cache-Control no-cache;  alias /tmp/jidometa/cache/;  try_files $uri @nocache; }

location @nocache {  rewrite    /cache/builds/(.*)-details.json /builds/$1/details break;  rewrite    /cache/builds.json /builds break;  proxy_pass http://jidometa; }

As an example, a request will first try to serve the /cache/builds.json from /tmp/jidometa/cache/builds.json - if the file doesn’t exist, it tries the @nocache directive. That directive will then send the request to /builds on the API, which will return a fresh response (and generate a new cached builds.json).

We also send the Cache-Control: no-cache HTTP header to ensure browsers don’t cache the cached response... but here’s the thing, browsers WILL cache the response regardless, except the no-cache directive will force the browser to use the Last-Modified or ETag headers in future requests, to revalidate whether the cached entry was changed or not. Well, assuming browsers handle those headers according to the RFC specs ;)

If the cached file hasn’t changed, the browser won’t fetch the entire file again.

Invalidating the cache

Builds are linked to their parent build (if any), as well as their child builds (if any).

When a certain build is modified (ex: the build’s status is changed from unreleased -> released), its cached response needs to be invalidated. We decided the simplest and easiest way to invalidate a cached entry is to delete the build’s cached response, and all the other responses linked to it (its parent and children).

Here’s the code we use for invalidation (PicoLisp):

(de remove-cached-file (Filename)  (call 'rm "-f" (pack "/tmp/jidometa/cache/" Filename)) )

(de remove-cached-build (Build)  (remove-cached-file (pack "builds/" Build "-details.json")) )

(de remove-all-cached-builds (Builds)  (mapcar '((N)            (remove-cached-build (; N 1)) )            Builds ) )

A call to (remove-all-cached-builds) with the list of Builds as the only argument, will handle removing each cached response individually. What’s nice is we can also use (remove-cached-file “builds.json”ˆ) to remove the entire builds list from the cache.

Of course, we wrapped all those calls in a simple (invalidate-cache) function which can be called anywhere in our code; anywhere that updates an existing build, or inserts a new one.

If a build has one parent and three child builds, then a total of six cached responses will be removed from /tmp/jidometa/cache/, including the builds.json (list of all builds). This is quite an aggressive cache invalidation strategy, but it ensures there are never any stale responses, and a new cached response will be regenerated as soon as a request is made for the “missing” file. Easy.

Issues with caching

We have yet to encounter any issues with cached responses, but if they do occur, it’s quite simple to identify exactly which response is problematic, as well as delete it manually from the cache directory. It’s also simple to completely wipe the cache from the system and start over from scratch. This happens after reboot anyways.

The main issue with a fresh cache is the first page load will be slightly slower than the others. It might take a few clicks to “warm” the cache with fresh responses, but considering the use-case for Jidometa (on-prem virtual appliance), we can live with that. Had there been a requirement to support hundreds of concurrent users, we would have considered an in-memory cache with persistence, such as Redis... maybe.

In conclusion

I would go into further detail, but really there’s nothing more to it. Serving cached JSON responses the way we do is somewhat of a “poor man’s solution”, but the lack of overhead and the simplicity of the implementation are a net win for everyone.

We’re beginning a series of articles to discuss the Jidometa internals.

We want to begin by talking about how Jidometa handles concurrency.

In the past...

Our initial SaaS service - Jidoteki - was built using Ruby, Redis, and Resque for concurrent builds. We quickly ran into issues when aiming to scale beyond one server. Those technologies were definitely great, but quite limiting in our ability to build across a cluster of servers (without implementing our own wacky scaling scheme - no thanks).

We replaced Resque with RabbitMQ, which served us extremely well for a few years. The wonderful administration interface, its somewhat low overhead, and the fact that all our apps (ruby, nodejs, bash) could easily talk to it were a plus.

The present..

With our move from SaaS to On-Prem appliance (Jidometa), we realized RabbitMQ would be overkill for such a deployment. We wanted to build an appliance with minimal dependencies, and a tiny footprint.

We chose to replace Redis and RabbitMQ, with SQLite and Mosquitto (MQTT).

Weird stack

I haven’t found any public articles on doing concurrency in SQLite using Mosquitto, so it was an adventure attempting to do it without guidance. Perhaps our stack is weird, considering we mostly write our code in PicoLisp now, but since we were already quite familiar with it, the decision was simple.

As many know, SQLite on its own is not ideal for write concurrency. Some best practices include: 

setting “PRAGMA journal_mode = wal;”

setting an initial connection timeout to a ridiculously high number (we set ours to 20 minutes)

simply NOT doing concurrent writes. Hah!

Doing concurrent writes

To solve the problem of concurrent writes in SQLite, we opted to serialize our database writes over an MQTT “queue”. I quote queue because it’s not technically a queue, although with the correct settings, it can be (mosquitto.conf):

allow_duplicate_messages false

max_inflight_messages 1

max_queued_messages 0

persistence true

Those are some options which will allow you to have a safe, FIFO-based MQTT “queue”. Since database writes are really important, it’s necessary to use QoS 2 on both ends (publisher and subscriber).

Our publisher and subscriber

This is where PicoLisp comes in. At the moment we’re not using a native library to access SQLite from within PicoLisp. We’re using a much less efficient system command: (call ‘sqlite3).

We wrote a subscriber daemon which listens for messages on the mosquitto/MQTT queue, and then processes the messages, validates the data, and performs the DB write. The messages can arrive concurrently, but only one message will ever be processed at a time - thus enabling serial DB writes. You can think of it as Unicorn with only 1 worker.

On the other end, the Jidometa API and backend scripts publish messages to the queue whenever something needs to be processed and written to the database. Those published messages can occur concurrently, at very high rates, without worrying about messages being missed, lost, or dropped (since it’s all occurring locally over localhost).

All DB reads are made directly to the SQLite database file, since reads can be concurrent and won’t affect the database in any way.

Big advantages

The added benefit of enabling MQTT behind the scenes, is the mosquitto server also ships with native built-in Websocks and TLS support.

We’ve opted to use that for reading and publishing build log messages, status updates, and other things which update the browser. On the browser side, we’re using a slightly modified version of the Paho MQTT JavaScript library. Our modifications include the ability to store messages in cookies, and making LocalStorage optional.

Moreover, we can easily extend the worker to perform other tasks when it receives certain messages, such as running integration scripts, performing basic maintenance, sending out alerts..

No locked database

Since implementing our concurrency in sqlite using mqtt, we’ve eliminated “database is locked” errors and are able to use the tiny file-based database the way we want.

Of course, we could have avoided this by using a different relational database (ex: PostgreSQL), but it wouldn’t have given us the wonderful properties of a tiny footprint, Websockets, and MQTT.