Task Queues in TypeScript: A Practical Guide

Working with Task Queues in TypeScript: In Practice

In modern web applications, there is often a need to perform long-running or resource-intensive operations asynchronously without blocking the main user thread. Sending email newsletters, generating PDF reports, processing uploaded images — all of these are ideal candidates for offloading to background tasks. A task queue is a proven architectural pattern that allows you to organize reliable execution of such operations. In this article, we will explore how to create and use task queues in TypeScript in practice, using popular libraries and approaches.

We will cover creating a simple queue, working with priorities, error handling, and integration with Redis via the BullMQ library. All examples will be written in TypeScript with an emphasis on strict typing.

1. Basics: Creating a Simple Task Queue

To start, let's implement a simple task queue without external dependencies. This will help understand the basic principles of operation. Our queue will store task functions and execute them sequentially.

type Task<T> = () => Promise<T>;\n\ninterface TaskItem<T> {\n  task: Task<T>;\n  resolve: (value: T | PromiseLike<T>) => void;\n  reject: (reason?: any) => void;\n}\n\nclass SimpleQueue {\n  private queue: TaskItem<any>[] = [];\n  private isProcessing = false;\n\n  add<T>(task: Task<T>): Promise<T> {\n    return new Promise<T>((resolve, reject) => {\n      this.queue.push({ task, resolve, reject });\n      this.process();\n    });\n  }\n\n  private async process(): Promise<void> {\n    if (this.isProcessing || this.queue.length === 0) return;\n    this.isProcessing = true;\n\n    const item = this.queue.shift()!;\n    try {\n      const result = await item.task();\n      item.resolve(result);\n    } catch (error) {\n      item.reject(error);\n    } finally {\n      this.isProcessing = false;\n      this.process();\n    }\n  }\n}\n\n// Usage\nconst queue = new SimpleQueue();\n\nqueue.add(async () => {\n  console.log('Task 1 started');\n  await new Promise(r => setTimeout(r, 1000));\n  console.log('Task 1 completed');\n  return 'Result 1';\n});\n\nqueue.add(async () => {\n  console.log('Task 2 started');\n  return 'Result 2';\n});

This example demonstrates the basic mechanics: tasks are added to an array and executed in order. Each task returns a Promise, which allows you to conveniently work with the results. However, this approach has limitations: no support for priorities, retries, or state persistence when the application restarts.

2. Prioritization and Error Handling

In real-world projects, tasks often have different levels of importance. For example, a transactional email sending task should be executed before a temporary file cleanup task. Let's add priorities to our queue.

type Priority = 'high' | 'medium' | 'low';\n\ninterface PriorityTask<T> {\n  task: Task<T>;\n  priority: Priority;\n  resolve: (value: T) => void;\n  reject: (reason?: any) => void;\n  retries: number;\n}\n\nclass PriorityQueue {\n  private high: PriorityTask<any>[] = [];\n  private medium: PriorityTask<any>[] = [];\n  private low: PriorityTask<any>[] = [];\n  private isProcessing = false;\n  private maxRetries = 3;\n\n  add<T>(task: Task<T>, priority: Priority = 'medium'): Promise<T> {\n    return new Promise<T>((resolve, reject) => {\n      const taskItem: PriorityTask<T> = {\n        task,\n        priority,\n        resolve,\n        reject,\n        retries: 0\n      };\n      this.getQueue(priority).push(taskItem);\n      this.process();\n    });\n  }\n\n  private getQueue(priority: Priority): PriorityTask<any>[] {\n    switch (priority) {\n      case 'high': return this.high;\n      case 'medium': return this.medium;\n