Task Queues in Java: From ThreadPoolExecutor to Disruptor

Task Queues in Java: From Built-in Mechanisms to Modern Frameworks

Task Queues are a fundamental pattern for building scalable and fault-tolerant Java applications. They allow you to separate business logic from thread management, balance load, and process tasks asynchronously. In this article, we will cover the main implementations of task queues in Java, their architecture, code examples, and usage scenarios.

1. Built-in Mechanisms: ThreadPoolExecutor and BlockingQueue

The standard Java library (java.util.concurrent) offers a powerful set of tools for working with task queues. The foundation is the ThreadPoolExecutor class, which manages a pool of threads and uses a queue to store pending tasks.

Basic Example:

import java.util.concurrent.*;

public class TaskQueueExample {    public static void main(String[] args) {        // Create a thread pool: corePoolSize=2, maxPoolSize=4, queue for 100 tasks        ThreadPoolExecutor executor = new ThreadPoolExecutor(            2, // corePoolSize            4, // maximumPoolSize            60, // keepAliveTime            TimeUnit.SECONDS,            new LinkedBlockingQueue<>(100) // task queue        );

        // Submit tasks        for (int i = 0; i < 10; i++) {            final int taskId = i;            executor.submit(() -> {                System.out.println("Task " + taskId + " is being executed by thread " +                                    Thread.currentThread().getName());                try {                    Thread.sleep(1000);                } catch (InterruptedException e) {                    Thread.currentThread().interrupt();                }            });        }

        // Graceful shutdown        executor.shutdown();    }}

Queue Types for ThreadPoolExecutor:

• LinkedBlockingQueue — an unbounded queue based on a linked list. Suitable for systems where the number of tasks is unknown in advance.
• ArrayBlockingQueue — a bounded queue based on an array. Allows you to control the maximum number of pending tasks.
• SynchronousQueue — a queue with no capacity: each put() waits for a take(). Used for direct handoff of tasks to threads.
• PriorityBlockingQueue — a priority queue. Tasks are sorted according to a given comparator.

2. Producer-Consumer Architecture Based on Queues

The classic Producer-Consumer pattern is easily implemented using BlockingQueue. Producers place tasks into the queue, and consumers retrieve them for processing.

Implementation Example:

import java.util.concurrent.*;

public class ProducerConsumerExample {    private static final BlockingQueue<String> queue = new LinkedBlockingQueue<>(5);

    static class Producer implements Runnable {        @Override        public void run() {            try {                for (int i = 0; i < 10; i++) {                    String task = "Task-" + i;                    queue.put(task); // blocks if the queue is full                    System.out.println("Produced: " + task);                    Thread.sleep(100);                }            } catch (InterruptedException e) {                Thread.currentThread().interrupt();            }        }    }

    static class Consumer implements Runnable {        @Override        public void run() {            try {                while (true) {                    String task = queue.take(); // blocks if the queue is empty                    System.out.printl