Working with Task Queues in C#: A Complete Guide

Introduction to Task Queues in C#

In modern multithreaded programming, the task queue is a fundamental pattern for organizing asynchronous data processing. In C# (.NET), this pattern is implemented through several powerful mechanisms that allow developers to efficiently manage execution threads and distribute workload.

The task queue solves the classic producer-consumer problem, where one part of the code creates tasks and another part processes them. In this article, we will look at the main ways to implement task queues in C#: from simple collections to modern channels (Channel).

Main Approaches to Implementing Task Queues

1. ConcurrentQueue — Thread-Safe Queue

The simplest and lowest-level implementation is ConcurrentQueue<T> from the System.Collections.Concurrent namespace. It provides thread-safe Enqueue and TryDequeue methods.

using System;using System.Collections.Concurrent;using System.Threading.Tasks;

public class SimpleTaskQueue{    private readonly ConcurrentQueue<Action> _tasks = new();    private readonly Task _worker;

    public SimpleTaskQueue()    {        _worker = Task.Run(ProcessTasks);    }

    public void Enqueue(Action task)    {        _tasks.Enqueue(task);    }

    private void ProcessTasks()    {        while (true)        {            if (_tasks.TryDequeue(out var task))            {                task();            }        }    }}

Advantages:

• Ease of use
• Low memory consumption
• Lock-free thread safety

Disadvantages:

• Busy waiting in the loop
• Lack of signaling for new tasks
• Difficulty in stopping

2. BlockingCollection — Queue with Blocking

BlockingCollection<T> solves the busy waiting problem. It provides Add and Take methods that block the thread until data becomes available.

using System;using System.Collections.Concurrent;using System.Threading.Tasks;

public class BlockingTaskQueue : IDisposable{    private readonly BlockingCollection<Action> _tasks = new(new ConcurrentQueue<Action>());    private readonly Task[] _workers;    private bool _disposed;

    public BlockingTaskQueue(int workerCount = 4)    {        _workers = new Task[workerCount];        for (int i = 0; i < workerCount; i++)        {            _workers[i] = Task.Run(ProcessTasks);        }    }

    public void Enqueue(Action task)    {        _tasks.Add(task);    }

    private void ProcessTasks()    {        foreach (var task in _tasks.GetConsumingEnumerable())        {            if (_disposed) break;            task();        }    }

    public void CompleteAdding()    {        _tasks.CompleteAdding();    }

    public void Dispose()    {        _disposed = true;        _tasks.CompleteAdding();        Task.WaitAll(_workers);        _tasks.Dispose();    }}

Key Features:

• Automatic thread blocking when no tasks are available
• Support for cancellation via CancellationToken
• Ability to limit queue capacity
• Safe completion via CompleteAdding

Modern Approach: Channel<T>

Starting with .NET Core 3.0 and .NET 5, C# introduced System.Threading.Channels.Channel<T> — this is the most performant and flexible implementation of a task queue. It supports asynchronous operations, cancellation, and throughput limitation.