Task Queues in Rust: A Practical Guide with Code Examples

Introduction: Why Task Queues in Rust?

In modern development of high-load systems, task queues are a fundamental pattern. They allow you to separate task generation from task execution, balance load, scale workers, and build fault-tolerant applications. Rust, with its strict memory control and lack of a garbage collector, provides unique opportunities for implementing efficient queues.

In this article, we will analyze three practical approaches to working with task queues in Rust:

\n
• Using standard mpsc (multi-producer, single-consumer) channels;
\n
• Multithreaded queues with the crossbeam library;
\n
• Asynchronous queues on tokio.
\n

Each approach will be accompanied by a working code example with explanations. You will learn how to choose the right tool for a specific task and avoid common mistakes.

1. Classic Channels: std::sync::mpsc

The Rust standard library provides the std::sync::mpsc module — a multithreaded queue with a single consumer. This is an ideal choice when you have multiple producers (threads that send tasks) and one worker (a thread that executes them).

Simple Example: Single Worker Pool

use std::sync::mpsc;\nuse std::thread;\nuse std::time::Duration;\n\nfn main() {\n    // Create a channel: sender (tx) and receiver (rx)\n    let (tx, rx) = mpsc::channel::<String>();\n\n    // Start a worker thread\n    let worker = thread::spawn(move || {\n        for task in rx {\n            println!("[Worker] Executing task: {}", task);\n            thread::sleep(Duration::from_millis(500));\n            println!("[Worker] Task '{}' completed!", task);\n        }\n    });\n\n    // Create several producers (in one thread for simplicity)\n    let producer = thread::spawn(move || {\n        let tasks = vec![\n            "Process file A",\n            "Send email",\n            "Update database",\n            "Generate report",\n        ];\n        for task in tasks {\n            println!("[Producer] Sending: {}", task);\n            tx.send(task.to_string()).unwrap();\n            thread::sleep(Duration::from_millis(200));\n        }\n    });\n\n    producer.join().unwrap();\n    // After the sender (tx) is dropped, the receiver (rx) will get None and the loop will end\n    drop(tx); // close the channel from the producer side\n    worker.join().unwrap();\n}

Features:

\n
• The mpsc channel is buffered — tasks do not block the sender until the buffer is full.
\n
• Only one receiver — this is a limitation, but it ensures simplicity and thread safety.
\n
• When all senders are dropped, the channel automatically closes.
\n

Multiple Workers: Manual Distribution

If you need multiple workers, mpsc is not directly suitable. The solution is to create one channel and several threads that will compete to receive tasks. However, this requires caution: Receiver does not implement Sync, so it needs to be wrapped in a Mutex or Arc.

use std::sync::{mpsc, Arc, Mutex};\nuse std::thread;\nuse std::time::Duration;\n\nfn main() {\n    let (tx, rx) = mpsc::channel::<String>();\n    let rx = Arc::new(Mutex::new(rx));\n\n    let mut workers = vec![];\n    for id in 0..3 {\n        let rx = Arc::clone(&rx);\n        let worker = thread::spawn(move || {\n            loop {\n                let task = {\n                    // Acquire the mutex and get the task\n                    let rx = rx.lock().unwrap();\n                    rx.recv()\n                };\n                match task {\n                    Ok(task) => {\n                        println!("[Work