Log Viewer Architecture
This can be one of the many architectures which can be used to develop an end to end solution for web UI to show real time application log.
Think of deployment and build tools.
This solution joins two distinct modern technologies to solve one problem of real-time log viewing.
There are two parts to the problem,
Writing Log:
Use Capped Mongo DB collection to store logs, where each document will be a log line.(you can use multiple collections created on the fly per job, think jenkins).
Since it is capped there is no need to clean up of old log files, or worry about disk space, but be diligent while deciding the cap.
Let your application write the log statements to this mongo, you can use a log appender if you use log4j or jdk logging.
you can also make #3 hit and forget from application POV, with a thread pool to handle inserts.
Reading Log
Main distinguishing feature of this architecture is in this part of it,
Here we use Tailable Cursor feature of mongodb and tie it to Web sockets connections.
Write a UI which starts a Web Socket connection to your ws servlet passing Job ID when a job is clicked.
Your servlet should maintain a Job ID to List of Connections Connection.
On Open connection, insert Connection in the above Map, start a thread which starts reading tailable cursor for that job id.
Tailable cursor is like "tail -f" command in linux, where the cursor waits without closing when there is no more data in the collection until new data is inserted.
So when ever new log statement is inserted by the application we cursor we opened gets data.
Since we have list of connection per job id, we just write messages to the ws connection once we get data from tailable cursor.
Few things to remember,
When connection is closed, remove it from the List.
if List of connections is 0 for a given job, close the tailable cursor and end that thread and delete the job from map.
Above architecture solves the problem of developing a web application to view realtime log.
This solution can be improved on above base structure in different ways to scale it, depending on your requirements.
Like,
Pre fetch and cache current active job log before any ws connection is made.
use normal cursors for non active jobs.
shard data based on job id.
Configure LB before your servlet, to route to singe node based on job id.
