Scaling Node on Multi-Cores System

Node.JS is one of the most popular languages for making web development applications. It is a platform built on Chrome’s Javascript v8 for easily building fast, scalable, network applications.

The most interesting thing about Node.JS is its single-threaded nature, which is a deliberate design decision by the Node.JS creator to avoid semaphore and/or mutex locking mechanisms that ultimately introduce complexity and hard-to-debug bugs into the code.

This also means that it only works and utilizes a single core of a system and all remaining cores remain idle. However, there are two most used ways which you can use to scale node.js on multi-core systems and I will be discussing them one by one.

Node Scaling on Multi-cores

Including ‘child_process’ module

The first way is to let resource allocation happen at the system level where incoming requests are distributed to multiple single-threaded Node.js processes each running in a virtual machine assigned a single core from the multi-core processor. All the instances live behind a proxy which serves to balance incoming requests to available Node processes.

Three ways of Creating Child Process

child_process.exec()

child_process.spawn()

child_process.fork()

1 exec()

Runs a command and buffers the output. It returns the buffer with the max size and waits for the process to end and tries to recall it.

2 spawn() Launches a new process with the given command. It should be used when the process returns a large amount of data. Spawn() starts receiving the response as soon as the process starts executing. It’s faster than the exec() call.

3 fork() It is the special case of spawn(), that returns an object with a built-in communication channel in addition to having all the methods in a normal child process instance.

Including Node ‘cluster’ module

The Node community has come forward with an even better second solution which breaks into multi-core awareness and capability natively into the Node.js platform.

Clustering Node.js runs as many dedicated single-threaded processes under a master

Node.js process without the need for elaborate virtual machine infrastructure, using fork() of cluster module. This results in significant performance gains without the costs associated with virtualization.

Performance Tests using Single and Multi-cores

Let's look at a performance test by our developer, in which he started the main process called the master process. The master process starts and manages other processes called worker processes which do the actual work of handling incoming requests. TCP connections are shared by all worker processes and a default load balancer is included transparently in the master Node.js process which serves to distribute the load.

This test was performed on a dual-core system with hyperthreading to take advantage of all four processing threads and made a simple app using Express generator and modified it by creating a processor-intensive root route to test performance under heavy load conditions.

Then run the following test using siege to measure the performance with and without clustering.

siege -c100 -t1M

Here siege is the HTTP benchmark by Node.JS, the c option instructs siege to throw 100 concurrent connections on the server whereas the t option instructs it to make up to 1 million total requests until the test is finished. The web server was running locally on port 3000 so the address of the server is localhost:3000.

Results

1. Using Single Core

Using Multi-cores

Conclusion

Now, we can say that the best practice of scaling Node is by using a cluster module, we noticed that the tangible benefits are achieved with scaling on multi-core using a cluster module on a single-threaded Node.JS application.