#rocksdb

And, roughly at the same time, deep inside Google they combined Log Structured Merge (LSM) Trees with Sorted String Tables (SSTables).

Now if you draw FT and LSM on a white-board side-by side and step back and squint they are actually fairly similar.  The dominating thing is that layers are hierarchical - the storage has the same shape and the IO is fairly equivalent.

There’s this pattern emerging where Google publish some ground-breaking utility code (often after they’ve already started on its replacement internally, it seems) and Facebook iterate and maintain it.  LevelDB became RocksDB.  Snappy got FB sponsoring LZ4 ZStd etc perhaps?  And so on.

And so RocksDB emerges as a well-supported, stable but also constantly improving LSM+SSTable that has excellent compression too.  The thing to underline here is that RocksDB is not about to be abandoned and bitrot; instead, its only going to get better!

Tokutek did not get the big success it deserved in the DB world, and eventually got brought by Percona.  And the TokuDB packaged by Percona or MariaDB using the constraining MySQL storage engine interface was not as world-beating as that originally integrated by Tokutek.  TokuDB was still my preferred storage engine though.  (previously: Compressing MySQL stats).  One thing that is exciting though is that Percona may finally integrate faster UPSERT hints - about time!!

Facebook have already integrated RocksDB into MySQL as myrocks.io.  I haven’t used it.  What has got me excited though is that Percona seem to be packaging it up in Percona-Server too!  Here’s a nice set of slides from a talk I wished I could have watched: TokuDB vs MyRocks

There is another general pattern in the DB world today: swap out your storage engine for RocksDB!  Just yesterday we saw the stunning results Instagram got with Cassandra doing it.  Even MongoDB went that way, and once upon a time TokuMX was vying for that pie.

So TokuDB may still be better than RocksDB for all kinds of technical reasons, but in the end, backing RocksDB is going to be less pain and technical debt longer-term.  It might even support foreign keys and other stuff that TokuDB never got around to.

I am quite sad for Tokutek’s plight and legacy though.