#mongo replication

Mongo records all operations that modify the data in a replica set in the oplog. Each entry in the oplog has a special timestamp called the optime.

The optime is a couple of integers: a first value which is a timestamp with precision to the second, and a 32 bit incremental value for operations "in the same second". So, basically, every time the first value increases, the second one restarts from zero.

Secondary members of the set will receive the optime from the primary with the statement they synchronize to, and store it unchanged. So, on secondaries the last optime states which operation has just been received from the primary and is now executing locally.

To get information on the state of a running mongod process, some external tools are available.

The only one bundled with MongoDB is the HTTP console, which provides a simple web interface listing information of interest to administrators. It can be accessed at the port with numeric value 1000 more than the configured mongod port: that is, for the mongod listening on port 27017, the console can be accessed at http://localhost:28018 in a web browser. Some of these information can be accessed by launching mongotop or running the db.currentOp() command in the shell.

By launching mongod with the additional --rest parameter, two more services are available on the same port. One is the Simple REST interface, which only supports data retrieval: no create, update or delete but it allows to issue any kind of query using URLs and returns the result as strict JSON. It can be found at http://localhost:28018/dbName/ where dbName is the name of the database to query.

Finally, there's the Replica Set Admin UI. For some reason, you also need to have ran your server with:

mongod --rest someOtheStuffHere

Clients issue write operations with some level of write concern, which describes the level of concern or guarantee the server will provide in its response to a write operation.

Available levels are:

Errors ignored: can be very efficient, but it's really too unsafe for normal operations.

Unacknowledged: errors are handled, but MongoDB will still not acknowledge the receipt of write operations.

Acknowledged: waits until mongod confirms the receipt of the write operation. With this level of write concern, clients can catch network, duplicate key, and other exceptions.

Journaled: the mongod instance will confirm the write operation only after it's been written to the journal. This ensures durability in case of a single instance database.

Replica Acknowledged: the write concern for replica sets has additional parameters. Most important, it allows to wait for a given number of members to give the receipt of a write before acknowledging to the client. Setting this to the special value majority avoids most rollback situations.

When a new primary is elected, it may miss some of the data the old primary had: the last writes might not have had the time to be replicated all over the set.

When the old primary comes back online (as a secondary) will use the oplog of the new primary to find about these spurious data and remove them, as well as import all the new data written in the meanwhile.

If too much time has elapsed since the failback this won't work: the oplog is a capped collection, that is old data is deleted after a certain size is reached. In this case, the whole data set will be copied from the primary.

By default, not only writes but even reads are only allowed from the primary.

Since there's only one primary, this allows for strong write consistency. This means that you won't read stale data: if you write something, you or another application will be immediately able to read that data.

Other NoSQL technologies advocate "eventual consistency", where this kind of delay between writing data and the same data being available to read is tolerated. This is supposed to give better performances.

In respect to electing a new primary, members of a replica set can have different roles:

Regular node: a normal node that has the data, can vote and become primary itself.

Secondary only: a node that has the data, can vote but not become primary itself.

Arbiter: a node with no data that is useful to make sure that you have the majority of votes in case of failover. Can only vote but not become primary.

Delayed node: a secondary node on which updates are replicated with a given delay, say one hour, so that it has the same data a regular node had one hour ago: useful to recover from disasters. Can vote but not become primary itself.

Database replication with MongoDB adds redundancy, helps to ensure high availability, simplifies certain administrative tasks such as backups, and may increase read capacity.

Replication is achieved via the creation of a replica set, a cluster of running mongod processes. On of these instances is designated as the primary, the rest are secondary members.

Clients direct all writes to the primary, while the secondary members replicate from the primary asynchronously.

When the primary fails, the remaining members will automatically try to elect a new one, as long as a majority of the original set members can still connect to each other: this condition is necessary to avoid that the replica set splits in two halves, each one electing its own primary and working independently.