Task Queues in JavaScript (Node.js): A Complete Guide for Beginners

Online Python Trainer for Beginners

Learn Python easily without overwhelming theory. Solve practical tasks with automatic checking, get hints in Russian, and write code directly in your browser — no installation required.

Start Course

Introduction: Why Do You Need Task Queues in JavaScript?

In modern web development, applications often encounter tasks that cannot be completed instantly within a single HTTP request. Sending emails, processing images, generating PDF reports, or synchronizing with external APIs — all of these take time and can "hang" the server if executed synchronously.

Task Queues solve this problem. They allow you to defer resource-intensive operations and process them asynchronously in the background. In this article, we will explore how to create and manage task queues in the Node.js environment using popular libraries.

\n\n

1. Basics of Working with Queues in Node.js

A task queue is a data structure that operates on the FIFO (First In, First Out) principle. Tasks are added to the end of the queue and processed from the beginning. In the context of Node.js, this means your server can quickly respond to the user ("Task accepted for processing"), while the task itself is executed later.

Specialized libraries are used to work with queues in Node.js, which rely on external storage (most often Redis). Let's look at the two most popular ones:

Bull — a modern, fast, and stable library based on Redis.
Kue — an older but still popular library with a built-in web interface.

We will focus on Bull, as it is the de facto standard for new projects.

\n\n

1.1 Installing and Connecting Bull

First, install the necessary dependencies:

npm install bull ioredis

Now let's create a simple queue:

const Queue = require('bull');\nconst myQueue = new Queue('emailQueue', {\n  redis: {\n    host: '127.0.0.1',\n    port: 6379\n  }\n});\n\n// Add a task to the queue\nmyQueue.add({\n  to: 'user@example.com',\n  subject: 'Welcome Email',\n  body: 'Welcome to our website!'\n});\n\nconsole.log('Task added to the queue');

In this example, we created a queue named emailQueue and added one task to it — sending an email. The task is not yet executed; it is simply waiting for its handler.

\n\n

2. Processing Tasks from the Queue

For tasks to start executing, we need a handler (consumer). It will "listen" to the queue and take tasks from it for processing.

Let's create a file worker.js:

const Queue = require('bull');\nconst emailQueue = new Queue('emailQueue', {\n  redis: { host: '127.0.0.1', port: 6379 }\n});\n\n// Handler function\nemailQueue.process(async (job) => {\n  const { to, subject, body } = job.data;\n  console.log(`Sending email to ${to}...`);\n  \n  // Simulating sending an email (in a real project, this would be nodemailer)\n  await new Promise(resolve => setTimeout(resolve, 2000));\n  \n  console.log(`Email "${subject}" sent to ${to}`);\n  return { status: 'sent', email: to };\n});\n\nconsole.log('Handler started and waiting for tasks...');

Run this script, and then add a task from the first example. You will see the task being processed.

\n\n

2.1 Error Handling and Retries

In real projects, tasks may fail (e.g., the SMTP server is unavailable). Bull provides a built-in retry mechanism:

emailQueue.add(\n  { to: 'user@example.com', subject: 'Important Email' },\n  { \n    attempts: 5,          // Number of attempts\n    backoff: { type: 'exponential', delay: 2000 } // Exponential backoff\n  }\n);\n\n// Handle failure in the handler\nemailQueue.on('failed', (job, err) => {\n  console.error(`Task ${job.id} failed: ${err.message}`);\n});

\n\n