C# (.NET) Task Queues: Channels and TPL Dataflow

Task Queues for C# (.NET): System.Threading.Channels and TPL Dataflow

When developing multithreaded applications in C# (.NET), there is often a need to organize asynchronous data transfer between producers and consumers. The classic Producer-Consumer pattern is solved using task queues. In the .NET ecosystem, there are two powerful tools for this: System.Threading.Channels (introduced in .NET Core 3.0) and the TPL Dataflow library (part of .NET since 4.5).

In this article, we will cover both approaches, their features, and provide practical code examples.

1. System.Threading.Channels — Lightweight and Fast Queue

The System.Threading.Channels namespace provides thread-safe, asynchronous queues optimized for high-throughput scenarios. Main types:

Channel<T> — abstract channel class.
Channel.CreateUnbounded<T>() — unbounded queue (risk of memory overflow).
Channel.CreateBounded<T>(int capacity) — bounded queue with backpressure control.
ChannelWriter<T> — interface for writing.
ChannelReader<T> — interface for reading.

Example: Producer-Consumer with Channel

Let's create a simple example where one producer generates numbers and one consumer processes them.

using System.Threading.Channels;using System.Threading.Tasks;

var channel = Channel.CreateUnbounded<int>();

// Producervar producer = Task.Run(async () =>{    for (int i = 0; i < 10; i++)    {        await channel.Writer.WriteAsync(i);        Console.WriteLine($"Produced: {i}");        await Task.Delay(100);    }    channel.Writer.Complete(); // Signal completion});

// Consumervar consumer = Task.Run(async () =>{    await foreach (var item in channel.Reader.ReadAllAsync())    {        Console.WriteLine($"Processed: {item * 2}");        await Task.Delay(200);    }});

await Task.WhenAll(producer, consumer);Console.WriteLine("Done!");

Advantages of Channels:

Minimal overhead (no additional object allocations).
Built-in support for async/await and IAsyncEnumerable.
Ideal for high-load systems (e.g., network packet processing).

2. TPL Dataflow — Powerful Processing Pipeline

The System.Threading.Tasks.Dataflow library provides more complex blocks for building pipelines and data processing networks. Main blocks:

BufferBlock<T> — simple queue.
ActionBlock<T> — block that performs an action on each element.
TransformBlock<TInput, TOutput> — transforms each element.
BatchBlock<T> — groups elements into batches.

Example: Pipeline with ActionBlock

Let's implement the same task: a producer sends numbers, a consumer processes them.

using System.Threading.Tasks.Dataflow;

var actionBlock = new ActionBlock<int>(async item =>{    await Task.Delay(200);    Console.WriteLine($"Processed: {item * 2}");}, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 2 });

// Producerfor (int i = 0; i < 10; i++){    await actionBlock.SendAsync(i);    Console.WriteLine($"Produced: {i}");    await Task.Delay(100);}

actionBlock.Complete();await actionBlock.Completion;Console.WriteLine("Done!");

Advantages of TPL Dataflow: