Queues and Background Tasks in Rust: Writing an Async Worker in an Hour

Queues and Background Tasks in Rust: Writing an Async Worker in an Hour

Have you ever wondered why your web service sometimes "lags" when a user clicks the "Send" button? Most often, heavy operations performed directly during request processing are to blame. Sending emails, generating PDFs, processing images, integrating with external APIs — all of this should live its own life, not block the response to the user. The solution is queues and background tasks.

And if you're used to solving this problem in Python with Celery or RQ, then Rust has a surprise in store for you. Rust is not just fast — it provides unique control over memory and threads, allowing you to write ultra-efficient workers that consume minimal resources. In this article, we'll break down how to implement queues and background tasks in Rust using modern libraries and approaches. You'll learn how to build a reliable task processing system that won't crash under load.

Why Rust for Background Tasks? Three Non-Obvious Advantages

Before diving into the code, let's figure out why you would use Rust for queues when there are proven Python solutions.

\n
• Performance without GC (garbage collector). Python workers can "freeze" during garbage collection pauses. Rust doesn't have this problem — you control every byte of memory. This is critical for high-load systems where every millisecond matters.
\n
• Asynchrony out of the box. Rust with tokio or async-std allows you to handle thousands of concurrent connections on a single thread. You can run hundreds of workers without fear of memory overflow.
\n
• Reliability. Rust's ownership system guarantees that you won't get a data race in a background worker. Errors are handled explicitly — no hidden exceptions.
\n

Now that we understand the "why," let's move on to the "how."

Choosing a Message Broker: RabbitMQ vs Redis vs NATS

A task queue is not just a list. It's a message broker that stores tasks and distributes them among workers. The Rust ecosystem has crates for all popular brokers.

RabbitMQ + lapin: The Classic in the World of Queues

RabbitMQ is a reliable beast for enterprise tasks. The lapin crate provides a clean async API. It's ideal if you need message routing, delivery guarantees, and complex topologies.

Redis + fred: Simplicity and Speed

Redis is a Swiss Army knife. Using fred (an async Redis client), you can create a queue via lists (BLPOP) or Pub/Sub. Redis is great for simple scenarios: "send a task — get a result."

NATS + async-nats: Lightweight Broker

NATS is a cloud-native broker with minimal latency. If you need to process millions of messages per second on small instances, NATS is your choice.

In this article, we'll focus on RabbitMQ, as it gives the most complete control. But the principles will be the same for any broker.

Writing a Queue in Rust: Step-by-Step Example

Let's create a simple system: a task generator (producer) and a worker (consumer). We'll process email sending tasks.

Step 1: Project Setup and Dependencies

Create a new project:

cargo new rust_queue_demo\ncd rust_queue_demo

Add dependencies to Cargo.toml:

[dependencies]\ntokio = { version = "1", features = ["full"] }\nlapin = "2.4"\nserde = { version = "1", features = ["derive"] }\nserde_json = "1"\ntracing = "0.1"\ntracing-subscriber = "0.3"\nanyhow = "1"

We use tokio as the async runtime, lapin for RabbitMQ, serde for task serialization, and tracing for logging.

Step 2: Define the Task Struct