Task Queues in JavaScript for Beginners: A Complete Guide

Introduction: Why Do You Need Task Queues?

Imagine you are running a web server on Node.js. A user sends a request to send 1000 emails. If you process this request synchronously, the server will "freeze" for several minutes — all other users will not be able to get a response.

How to solve this problem? The answer is task queues. They allow you to defer the execution of resource-intensive operations (sending emails, generating PDFs, processing images) and run them in the background without blocking the main application thread.

In this article, we will cover:

• What a task queue is and how it works
• Popular libraries for Node.js (Bull, Kue)
• Code examples: creating, adding, and processing tasks
• Best practices for beginners

1. Basics of Task Queues in Node.js

What is a Task Queue?

A task queue is a data structure that operates on the FIFO principle (First In, First Out). Tasks are added to the end of the queue and processed from the beginning. Queues are typically built on top of Redis, a fast in-memory key-value store.

Main components of a queue:

• Producer — the code that adds tasks to the queue
• Consumer (Worker) — the code that retrieves and executes tasks
• Queue Manager — the mechanism that manages the queue (usually Redis)

Popular Libraries

There are two main libraries for Node.js:

• Bull — modern, fast, with support for priorities, retries, and delays
• Kue — an older but simple library (currently less popular)

In this article, we will focus on Bull, as it is actively maintained and has more features.

2. Installing and Configuring Bull

Step 1: Installing Redis

Bull uses Redis. If you don't have Redis, install it:

# macOSbrew install redis

# Ubuntusudo apt-get install redis-server

# Start Redisredis-server

Step 2: Installing Bull

Create a new project and install the package:

mkdir queue-democd queue-demonpm init -ynpm install bull

Step 3: Basic Configuration

Create a file queue.js:

const Queue = require('bull');

// Create a queue named 'emailQueue'const emailQueue = new Queue('emailQueue', {  redis: { host: '127.0.0.1', port: 6379 }});

module.exports = emailQueue;

3. Example: Queue for Sending Emails

Step 1: Creating a Producer (Adding Tasks)

Create a file producer.js:

const emailQueue = require('./queue');

async function addEmailTask(to, subject, body) {  // Add a task to the queue  const job = await emailQueue.add({    to: to,    subject: subject,    body: body  }, {    // Task options    attempts: 3,          // Number of retry attempts on error    backoff: 5000,        // Delay between attempts (5 sec)    delay: 1000           // Delay before execution (1 sec)  });

  console.log(`Task added with ID: ${job.id}`);  return job;}

// Example usageaddEmailTask('user@example.com', 'Hello!', 'Test message');

Step 2: Creating a Worker (Processing Tasks)

Create a file worker.js:

const emailQueue = require('./queue');

// Function simulating email sendingasync function sendEmail(to, subject, body) {  console.log(`Sending email to ${to}...`);  // Simulate delay  await new Promise(resolve => setTimeout(resolve, 2000));  console.log(`Email sent: ${subject}`);}

// Queue handleremai