#realtimeweb

What makes Node.js different from other platforms is in how it approaches I/O. It uses an event-loop in conjunction with asynchronous I/O, which allows it to achieve a high level of concurrency with a light footprint. 

Typically, when a program needs some sort of external input, it does so in a synchronous fashion. The following line of code should be very familiar to any programmer:

var results = db.query("SELECT * FROM users"); print(results[0].username);

All we are doing here is querying a SQL database for a list of all users, and then we are printing out the first user's name. When querying a database like this, there are many intermediary steps that need to be taken, such as:

Opening a connection to the database server.

Transmitting the request over the network to that server.

The server itself needs to process the request after receiving it.

The server must transmit the response back over the network to our application.

This list does not cover all the specifics, as there are many more factors than are necessary for the point to be made. By looking at our source code, this is treated as an instantaneous action, but we know better. We often neglect this wasted time because it is so fast that we don't notice it happening. Consider the following:

The Cost of I/O L1-cache 3 cycles L2-cache 14 cycles RAM 250 cycles Disk 41,000,000 cycles Network 240,000,000 cycles

Each I/O operation has a cost, which is paid directly in a program that uses synchronous I/O. There could easily be millions and millions of clock cycles that occur before the program can progress.

When writing an application server, a program like this can only serve one user at a time, and the next user cannot be served until all the I/O and processing is complete for the previous user. This is unacceptable of course, so the easiest solution is to create a new thread for each incoming request, so they can run in parallel.

This is how the Apache web server works, and it is not difficult to implement. However, as the number of simultaneous users increase, the amount of memory used also increases. Each of those threads requires overhead at the operating system level, and it adds up pretty quickly. In addition, the overhead of context switching between those threads is more time consuming than desired, further compounding the problem.

The nginx web server uses an event loop at its core to handle processes. By doing so, it is able to handle more simultaneous users at once, with fewer resources. An event loop requires that the bits of processing be broken up into small pieces, and run in a single queue. This removes the high cost of creating threads, switching back and forth between those threads, and requires less demand of the overall system. At the same time, it fills in the processing gaps, particularly those that occur during the wait for I/O to complete.

Node.js takes the event-driven model that nginx uses to such great success, and it exposes that same capability for many types of applications. In Node.js, all I/O is entirely asynchronous and does not block the rest of the application thread. The Node.js API accepts function parameters (usually known as a "callback function") for all I/O operations. Node.js then fires off that I/O operation, and lets another thread outside the application do the processing. After that, the application is free to continue handling other requests. Once the requested operation is complete, the event-loop is notified, and the callback function is invoked with the results.

As it turns out, waiting for I/O to complete is the most expensive part of many applications in terms of raw processing time. With Node.js, the time spent waiting for I/O is completely detached from the rest of your application's processing time. Your application just uses callback functions to process results as simple events, and JavaScript's ability to use closure retains the function's context, despite being executed asynchronously.

If you were to take up the task of writing a multi-threaded application, you would have to concern yourself with concurrency problems like deadlocks, which are very difficult (if not impossible) to reproduce and debug in real-world applications. With Node.js, your primary application logic runs on a single thread, free of such concurrency problems, while the time-consuming I/O is handled on your behalf by Node.js.

Like any other platform, Node.js has an API developers can use to write their applications. JavaScript itself lacks a standard library, particularly for performing I/O. This actually turned out to be one of the reasons that Ryan Dahl chose JavaScript. As the core API can be built from the ground up, without needing to worry about creating conflicts with a standard library, in case it is done wrong (given JavaScript's history, this is not an unreasonable assumption).

That core library is minimalistic, but it does include modules for the essentials. This includes, but is not limited to: filesystem access, network communication, events, binary data structures, and streams. Many of these APIs, while not difficult to use, are very low-level in implementation. Consider this "Hello World" demonstration straight from the Node.js website (with comments added):

// one of the core modules var http = require('http'); // creates an http server, this function is called for each //request http.createServer(function (req, res) { // these parameters represent the request and response //objects // the response is going to use a HTTP status code 200 //(OK) // the content-type HTTP header is set as well res.writeHead(200, {'Content-Type': 'text/plain'}); // lastly, the response is concluded with simple text res.end('Hello World\n'); }).listen(1337, '127.0.0.1'); console.log('Server running at http://127.0.0.1:1337/');

This server uses the http core module to set up a web server that simply sends "Hello World" to anyone who makes a request of it. This is a simple example, but without comments, this consists of only six lines of code in all.

The Node.js team has opted to keep the core library limited in scope, leaving the community of developers to create the modules they need for everything else, such as database drivers, unit-testing, templating, and abstractions for the core API. To aid in this process, Node.js has a package manager called npm.

npm is the tool that handles installing dependencies for Node.js applications. It opts for locally bundled dependencies, rather than using a single global namespace. This allows different projects to have their own dependencies, even if the version varies between those projects.

In addition to allowing for the use of third-party modules, npm also makes contributing to the registry a public affair. Adding a module to the registry is as simple as a single command, making the barrier to enter extremely low. Today, the npm registry has over 42,000 packages listed and is growing faster by the day. 

With the registry growing so fast, it's obvious there is a vibrant ecosystem behind it. I can personally attest to the fact that the Node.js developer community is very friendly, extremely prolific, and has an enormous amount of enthusiasm.

For TV advertisers having access to realtime and granular web analytics ensures that accurate analysis or spot matching can be performed to measure performance of certain spots or view % uplift from TV advertising.

Some web reporting tools like Google Analytics only supply volume of web traffic 'per hour' and other tools that do provide more granular results can be expensive. 

With our new 'Web Tracking' service Adalyser subscribers benefit from automated real time web stats that feed straight into their campaign reporting.

This gives users even greater insight to TV advertising but also delivers detailed web analytics in one easy to use tool.

If you are a current Adalyser subscriber and would like to find out more please get in touch with your account manager.

If you would like to find out more about becoming an Adalyser subscriber please get in touch +44 (0) 333 666 7366 / [email protected]

Adalyser is developed by OneSoon Limited. Advertising agencies and advertisers around the world use Adalyser to analyse data, visualise results, create bespoke reports, dashboards and optimise advertising investment.

I'm sitting in a restaurant. I pick up the menu and my choice is made: Eggs Benedict it is. The waiter approaches – he's called Dave – and I place my order. Dave dutifully records my choice, smiles and retreats to the kitchen. 

With little more than a grunt, Hans (the chef) takes the order from Dave and sets to work on my breakfast. He’s done this a million times.

After just a few minutes, Hans has finished; “Service”. Dave returns and in one swift motion the eggs benedict pass from hot plate to waiter's arm and begin their journey back to my table. 

I pretend I don't notice him re-entering the dining room but secretly I'm thrilled. As Dave approaches the table, my silent mental preparation gives way to the physical. Knife, fork: armed. Dish placed. “Thanks. This looks lovely.” And it is.

Restaurant service reminds of the web. And by the web I mean the original sir-tim-at-cern-web-as-pages-connected-by-hyperlinks version. Substitute me for the user, Dave for the browser & Hans for the server and you have a stupidly tenuous analogy for the experience of loading a web page.

Tenuous it may be but both processes – ordering food in a restaurant, requesting a page from a web server – have one thing in common. They are asynchronous. Each step of the process has to wait until the preceding one has finished before it can start.

In a restaurant the ‘wait’ is part of the experience. I'd go as far as to say I enjoy this. There's an element of theatre to the whole thing wouldn’t you say?

Of course on the web, the wait is over in split seconds. But it is undeniably part of the experience. Rendering engines get faster, bandwidth gets wider, but there remains a momentary pause between clicking that submit button and the confirmation page loading. Watch closely. It's surprising how often you see a blank screen between pages, albeit for a millisecond or two.  It may not be ‘by design’, but I'd speculate that this is still the most common form of feedback on the web. We've become accustomed to it and in a fairly crass way, it works.

Fast forward to the 'realtime' web and things have subtly changed. Pages load and the lack of an obvious page refresh means I don't even notice it. Pretty snazzy I guess. But you know what, I don't mind the gaps. It gives the web rhythm. 

Still, there’s lots of interaction design to be explored. If the native feedback mechanisms of the browser (blank page, loading animation) aren't enough, we're going to have to design these ourselves. Something we seem to be pretty bad at so far. iOS has helped demonstrate the gaps between pages (or states) are important to the experience. The transitions are subtle, tasteful and have an element of physicality, but only designers pay attention to them. For everyone else they're just the seams, the edges of the experience and unbeknown to them, feedback.