The single-threaded nature of Node.js

Node.js is a platform designed for fast and scalable backend development. Unlike some other backend frameworks, it operates on a single-threaded model, which may come as a surprise to some. Node.js has one main thread for performing all operations. In other words Node.js can do one job at a time. So, how is it possible for Node.js to handle hundreds or thousands of requests per second?

Most of the time while the HTTP server is working, there is nothing for the CPU to do because there are lots of external calls like saving objects to a database or fetching data from a cloud provider. Why should the CPU just wait and do nothing? Under the hood, Node.js can manage this problem and handle a next request while waiting for responses from external services. There is a mechanism called an ‘Event Loop’ that will be covered later in this blog post. Node.js can delegate tasks to other services that indeed can be multithreaded and perform multiple jobs at the same time, such as I/O operations (the OS can handle them) or database calls (the database engine can handle them).

Think of Node.js as a waiter in a restaurant. The waiter takes an order and hands it to the chef for preparation. In the meantime, while the meal is being prepared by the chef (i.e. a call to the database is being made), the waiter can take the next order. It’s worth mentioning that there can be more than one chef, and that would represent the ability of external services to perform multiple operations at the same time. The waiter does not need to wait until the meal is ready before attending to another table, but instead gets informed when an order is ready and can be served. The same goes for Node.js – it does not need to wait until the end of a long request to an external service, but can handle other requests in the meantime and simply receive a notification when the long one has been completed.

Because of this ability to perform time-consuming operations in the background, Node.js can be very fast and handle many requests simultaneously. Its single-threaded environment also means that a developer does not need to worry about managing threads on their own and all of the bugs like deadlocks, etc., can be avoided.

Event loop, blocking, and non-blocking

The Event Loop is the mechanism that allows Node.js to execute code asynchronously. Asynchronous operations performed in Node.js can have a callback, which is the function that should be run when an asynchronous call ends. This callback is passed to the Event Loop. Every iteration of the Event Loop has a few cycles that check if asynchronous calls were resolved, and the callback passed to it should be run by Node.js.

Node.js can delegate I/O operations to the OS Kernel, which of course can be multi-threaded. Because of that, Node.js can perform I/O operations in a non-blocking way. When an I/O operation is ended by the OS, it notifies the Event Loop to add an appropriate callback to the queue and Node.js finally executes it.

There are some downsides of the single-threaded approach that should be treated carefully. Node.js can handle asynchronous operations very well by putting them into the Event Loop and managing them when the data is ready. However, synchronous I/O operations are considered blocking because the main thread cannot operate on the other tasks – it needs to wait until, for example, the file will be read. It is very important to treat I/O operations asynchronously and not block the main thread. We also want to prevent CPU-heavy operations like data encryption from being called synchronously.

Let’s consider the situation when we have an HTTP server in Node.js: there is an endpoint that saves a user to the database. It doesn’t do much else, just validates the data and saves it. The request is made in 100 ms: the database call lasts 90 ms and the rest takes 10 ms. That means the CPU is busy for only 10 ms. So, theoretically, if we made the request to the database synchronously and blocked the main thread we would lose 90 ms in which the CPU is free and Node.js could handle other requests. It is very important to remember to make operations asynchronously.

Reading a file this way is considered blocking:

const data = fs.readFileSync("./test.txt");
console.log(data.toString());
console.log("After sync read");

The first line blocks the main thread while Node.js waits for the file to be read. Once it is done, the content is printed to the console. Finally, the string "After sync read" is displayed.

This is how asynchronous reading would look like:

fs.readFile("./test.txt", (err, data) => {
 if (err) throw err;
 console.log(data.toString());
});
console.log("After async read");

An I/O operation is handed over to the operating system and the callback provided to the readFile function is added to the Event Loop. The main thread continues to execute subsequent operations without being blocked. When the OS finishes reading the file, it informs the Event Loop and the callback is queued and will be executed as soon as Node.js is able to do so. The output will first display "After async read" and then the content of the file.

Promise, async/await, Promise.all

Promise is an object that represents a value, which will eventually be resolved or failed. At the moment of making an asynchronous call, we can’t be sure if it will complete successfully or not. Promise has three possible states: pending, fulfilled, or rejected.

Let’s analyze this piece of code:

const main = () => {
 dbCall(1000)
   .then((user) => {
     console.log("Database call ended");
     awsCall(user, 1000)
       .then(() => {
         console.log("AWS call ended");
       });
    });
};

Imagine a functionality that enables a user to generate some document for themself and save it on AWS S3. Firstly, we fetch the user's data from the database, generate the document (it is not included in this example), and then we save it on AWS. We simulate some time-consuming calls with asynchronous functions: dbCall and awsCall. The call to the database needs 1,000 ms to be resolved. After that, Node.js is notified that Promise was resolved. We can grab its potential value and do whatever we want with it by using the then function and passing a callback to it. Then, we call AWS and also pass a callback to the then function. That could lead to something known as “callback-hell”, in which there is a callback in a callback in a callback, etc. This reduces the readability of the code. Note that, in the above example, errors are completely ignored.

Help to the above problem comes the async/await syntax. The async keyword can mark a function asynchronous and await tells Node.js that it should wait for the result of the call instead of processing the next lines immediately. Due to that, we can read the code in a more synchronous way. Bear in mind that these functions are asynchronous, which means they are non-blocking and Node.js can process other requests while waiting. An asynchronous piece of code with the async/await syntax instead of then can be written as:

const main = async () => {
 const user = await dbCall(1000);
 console.log("Database call ended");

 await awsCall(user, 1000);
 console.log("AWS call ended");
};

It will do the same thing whilst also being much cleaner.

The code above will execute in approximately 2,000 ms. Imagine that an AWS call does not need any user information and can be made in parallel to a database call. Do we need 2,000 ms to perform these calls? Certainly not. We can avoid waiting for a database call to be made in order to call AWS and make these requests in parallel.

const main = async () => {
 await Promise.all([
   dbCall(1000),
   awsCall(1000)
 ]);
};

Promise.all allows us to wait for a few asynchronous calls at once and returns all of their results.

Synchronous vs asynchronous code – a benchmark

Let’s consider the following code:

const server = http.createServer(async (req, res) => {
 const salt = 10;

 if (req.url === "/sync") {
   const hash = bcrypt.hashSync("Polcode wins", salt);
   res.end(hash);
 }

 if (req.url === "/async") {
   const hash = await bcrypt.hash("Polcode wins", salt);
   res.end(hash);
 }
});

There is a simple HTTP server with two endpoints, "/sync" and "/async", which uses the bcrypt library to encrypt data. The performance of each endpoint will be measured using the ApacheBench tool. The "/sync" endpoint performs encryption synchronously in a blocking manner, while the "/async" endpoint performs encryption asynchronously in a non-blocking manner. Bcrypt uses C++ for encryption operations, which are CPU-intensive. Once the encryption is completed, the resulting hash is sent to the user. 500 requests will be sent to each endpoint, with 10 requests made at a time.

“/sync”:

“/async”:

Let’s compare both benchmarks:

It’s clear that encrypting data in an asynchronous (non-blocking) way is performed faster and more requests can be made per second.

The main difference between synchronous and asynchronous request processing lies in the fact that synchronous operations block the main thread, hindering Node.js from performing any other actions. In contrast, asynchronous operations utilize a callback-based approach in which the requested value is awaited and a callback sent to the Event Loop. This allows Node.js to handle subsequent requests while waiting for the value to be resolved.

Scaling

In terms of scaling, it’s easier to scale applications horizontally (increase the number of machines where an application runs) than vertically (increase the power of the machine) because vertical scaling has its limitations – we cannot scale vertically ad infinitum, and it’s less cost-efficient. Node.js scales horizontally pretty well. We just need more instances of our application and a Load Balancer to balance traffic between instances. It’s worth mentioning that Node.js can use the cluster module to spawn a few processes on the same machine and listen on the same port. That could be an improvement for our application when there are heavy CPU computations, but if we have a traditional web server handling many disk and network operations then it might not help a lot.

Conclusion

Node.js is a highly regarded platform for building fast and scalable backend web servers using Javascript or Typescript. It’s important to remember the single-threaded nature of Node.js and that it can use other multi-threaded services to perform simultaneous operations and non-blocking I/O – thanks to the Event Loop. Operations where possible should be performed asynchronously in a non-blocking way, preferably with the async/await syntax. Promise.all should also be considered when there is a need to perform a few asynchronous operations at the same time.