Scaling Node on Multi-Cores System

By May 3, 2016 MEAN No Comments

Node.JS is one of the most popular languages for making web development applications. It is a platform built on Chrome’s Javascript v8 for easily building fast, scalable, network applications.

The most interesting thing about Node.JS is its single-threaded nature, that is a deliberate design decision by the Node.JS creator to avoid semaphore and/or mutex locking mechanisms that ultimately introduce complexity and hard to debug bugs into the code.

This also means that it only works and utilizes a single core of a system and all remaining cores remain idle. However, there are two most used ways which you can use to scale node.js on multi-core systems and i will be discussing them one by one.

Node Scaling on Multi-cores

  • Including ‘child_process’ module

The first way is to let resource allocation happen at the system level where incoming requests are distributed to multiple single-threaded Node.js processes each running in a virtual machine assigned a single core from the multi-core processor. All the instances live behind a proxy which serves to balance incoming requests to available Node processes.

Three ways of Creating Child Process

  1. child_process.exec()
  2. child_process.spawn()
  3. child_process.fork()

1 exec()
Runs a command and buffer the output. It returns the buffer with the max size and waits for the process to end and tries to recall it.

2 spawn()
Launches a new process with given command. It should be used when the process returns large amount of data. Spawn() starts receiving the response as soon as the process starts executing. It’s more faster than the exec() call.

3 fork()
It is the special case of spawn(), that returns object with a built-in communication channel in addition to having all the methods in a normal child process instance.

  • Including Node ‘cluster’ module

The Node community has come forward with an even better second solution which breaks in multi-core awareness and capability natively into the Node.js platform.

Clustering Node.js runs as many dedicated single-threaded processes under a master Node.js process without the need for elaborate virtual machine infrastructure, using fork() of cluster module. This results in significant performance gains without the costs associated with virtualization.

Performance Test using Single and Multi-cores

Lets look at a performance test by our developer, in which he start the main process called the master process. Master process starts and manages other processes called worker processes which do the actual work of handling incoming requests. TCP connections are shared by all worker processes and a default load balancer is included transparently in the master Node.js process which serves to distribute the load.

This test was performed on a dual core system with hyperthreading to take advantage of all the four processing threads and made a simple app using Express generator and modified it by creating a processor intensive root route to test performance under heavy load conditions.

Then ran the following test using siege to measure the performance with and without clustering.

siege -c100 -t1M

Here siege is the http benchmark by Node.JS, the c option instructs siege to throw 100 concurrent connections on the server whereas the t option instructs it to make upto 1 million total requests until the test is finished. The web server was running locally on port 3000 so the address of the server is localhost:3000.

Results

  • Using Single Core

First review on the using single core
Transactions: 1237 hits
Availability: 100.00 %
Elapsed time: 59.70 secs
Data transferred: 0.03 MB
Response time: 4.14 secs
Transaction rate: 20.72 trans/sec
Throughput: 0.00 MB/sec
Concurrency: 85.68
Successful transactions: 1237
Failed transactions: 0
Longest transaction: 4.58
Shortest transaction: 0.11

  • Using Multi-cores

A noticeable performance is shown when we test it on multi-core
Transactions: 2523 hits
Availability: 100.00 %
Elapsed time: 59.92 secs
Data transferred: 0.06 MB
Response time: 1.82 secs
Transaction rate: 42.11 trans/sec
Throughput: 0.00 MB/sec
Concurrency: 76.67
Successful transactions: 2523
Failed transactions: 0
Longest transaction: 2.86
Shortest transaction: 0.12

node-js-response_time

node-js-transaction_rate

node-js-total_transaction

Conclusion

Now, we can say that the best practice of scaling Node is by using cluster module, we noticed that the tangible benefits are achieved with scaling on multi-core using cluster module on a single-threaded Node.JS application. You can get the example project on the link attach below

https://github.com/chaudhryjunaid/multicore-demo