#polyglot persistence

The Rise of NoSQL & Polyglot Persistence (by MarakanaTechTV)

Noteworthy Blog Posts of the Month

Password Authentication with Mongoose

Implementing Mass Login Attempts with Mongoose

Motor Progress Report GridFS

MongoDB and Hardware Profiles

MongoDB Schema Design at Scale

How MongoDB Makes Custom ecommerce Easy

Design and Performance Tips for MongoDB Indexes

New Driver Releases

MongoDB 2.2.1 rc0

Java Driver 2.9.0

Perl Driver 0.46.1

MongoDB 2.2 Round Up

MongoLab Aggregation Tutorial

Interesting Changes in MongoDB 2.2 You Might Not Know About

Controlling Collection Distribution with Tag Aware Sharding

Good Night Westley: Time to Live Collections

Using the MongoDB Aggregation Framework in Grails

Pedro Visintin:

We can distinguish several objects: Session, Cart, Item, User, Order, Product and Payment. Usually we use ActiveRecord to store all of them. But this time let’s think about it differently.

For sessions, we don’t need durable data at all — Redis can be a good option, and of course will be faster than any RDBMS. For Cart and Item,we will need high availability across different locations. Riak can fit well for this use case. For User Order Product and Payment, a relational database can fit well, focusing on Transactions and Reporting about our application.

This is a very good exercise for understanding the requirements for your data service layer. As much as I write about polyglot persistence, when architecting an application never leave aside or ignore the operational requirements for your service.

After learning about LinkedIn’s Databus low latency data transfer system, I’ve had a short chat with Sid Anand focused on understanding what makes Databus unique.

As I’ve mentioned in my post about Databus, Databus looks at first as a data-oriented ESB. But what is innovative about Databus comes from decoupling the data source from the consumers/clients thus being able to offer speed to a large number of subscribers that are up-to-date, but also help clients that fall behind or are just bootstrapping without adding load on the source database.

Databus clients are smart enough to:

ask for Consolidated Deltas since time T if they fall behind

ask for a Consistent Snapshot and then for a Consolidated Delta if they bootstrap

and Databus is build so it can serve both Consolidate Deltas and Consistent Snapshots without any impact on the original data source.

Diagram from Highscalability.com

The “catching-up” and boostrapping processes are described in much more details in Sid Anand’s article.

Databus is the single and only way that data is replicated from LinkedIn’s databases to search indexes, the graph, Memcached, Voldemort, etc.

Marting Fowler and Pramod Sadalage in an infographic promoting their upcoming book (PDF):

Polyglot persistence will occur over the enterprise as different applications use different data storage technologies. It will also occur within a single application as different parts of an application’s data store have different access characteristics.

There are over 2 years since I’ve begun evangelizing polyglot persistence. By now, most thought leaders agree it is the future. Next on my agenda is having the top relational vendors sign off too. Actually, I’m almost there: Oracle is promoting an Oracle NoSQL Database and Microsoft is offering both relational and non-relational solutions with Azure. They just need to say it.