Task Queues in Rust: A Complete Guide for Beginners

Introduction: Why Do You Need Task Queues in Rust?

When developing applications in Rust, there is often a need to process tasks asynchronously or distribute the load across multiple threads. A task queue is a fundamental data structure that allows you to organize sequential or concurrent processing of units of work. In this article, we will break down how to create and use task queues in Rust, starting with simple examples and ending with real-world scenarios.

Rust offers several approaches to implementing queues: from standard channels in std::sync::mpsc to advanced asynchronous solutions based on tokio. We will look at each of them and show how to choose the right tool for your task.

Basics: Task Queue via mpsc Channels

The simplest way to create a task queue in Rust is to use a multi-producer, single-consumer channel (mpsc) from the standard library. This mechanism is ideal for situations where one thread processes tasks sent from multiple producer threads.

Creating a Simple Queue

Let's look at a basic example where the main thread sends tasks and a separate thread processes them:

use std::sync::mpsc;use std::thread;use std::time::Duration;

fn main() {    // Create a channel: sender and receiver    let (tx, rx) = mpsc::channel::<String>();

    // Start a handler thread    let handler = thread::spawn(move || {        for task in rx {            println!("Processing task: {}", task);            thread::sleep(Duration::from_millis(500));        }    });

    // Send tasks    let tasks = vec!["Task 1", "Task 2", "Task 3"];    for task in tasks {        tx.send(task.to_string()).unwrap();        println!("Sent: {}", task);    }

    // Close the channel so the handler finishes    drop(tx);    handler.join().unwrap();}

This code demonstrates the basic mechanics: tx.send() places a task in the queue, and the receiver rx extracts them in the order they arrive (FIFO). Note that the channel automatically blocks the producer if the queue is full, and blocks the consumer if the queue is empty.

Multiple Producers

One of the main advantages of mpsc is the ability to clone the sender for multiple threads:

use std::sync::mpsc;use std::thread;

fn main() {    let (tx, rx) = mpsc::channel::<u32>();    let mut handles = vec![];

    // Start 3 producer threads    for id in 0..3 {        let tx_clone = tx.clone();        handles.push(thread::spawn(move || {            for i in 0..5 {                let task = id * 100 + i;                tx_clone.send(task).unwrap();            }        }));    }

    // Close the original sender    drop(tx);

    // Process all tasks    for received in rx {        println!("Received task: {}", received);    }

    for handle in handles {        handle.join().unwrap();    }}

Each producer sends its own tasks, and a single consumer processes them in the order they arrive. This is the classic "producer-consumer" pattern.

Advanced Queue with Priorities

The standard mpsc channel does not support priorities — tasks are processed strictly in the order they are sent. However, we can implement a priority queue using BinaryHeap from the standard library.

Implementing a Priority Queue

use std::collections::BinaryHeap;use std::cmp::Ordering;

#[derive(Debug, Eq, PartialEq)]struct Task {    priority: u8,    data: String,}

impl Ord for Task {    fn cmp(&self, other: &Self) -> Ordering {        // The smaller the number, the higher the priority        other.priority.cmp(&self