Task Queues in PHP: A Guide to Implementation and Optimization

Working with Job Queues in PHP: From Simple to Complex

In modern web applications, there is often a need to perform resource-intensive or long-running operations without making the user wait. Sending email notifications, generating reports, processing images, integrating with external APIs — all these tasks are ideal for asynchronous execution. This is where job queues come to the rescue. In this article, we will take a detailed look at how to implement and optimize working with queues in PHP, starting with simple solutions and ending with industrial-grade tools.

What is a Job Queue and Why Do You Need It?

A job queue is a mechanism that allows you to postpone the execution of some work for later. Instead of executing a task immediately within an HTTP request, you place it in a queue. A special worker or a group of workers retrieve tasks from the queue and process them in the background.

Main advantages of using queues:

• Faster response to the user: The user receives a response immediately, while the heavy work is performed asynchronously.
• Scalability: You can run multiple workers for parallel task processing.
• Reliability: If a task fails (e.g., due to a temporary error), it can be retried later.
• Offloading the main server: Background processes do not block the processing of new HTTP requests.

Simple Queue Based on a Database

The simplest way to organize a queue in PHP is to use a database table (e.g., MySQL). This approach does not require installing additional software and is perfect for small projects.

Step 1: Creating the jobs table

CREATE TABLE jobs (    id INT AUTO_INCREMENT PRIMARY KEY,    queue_name VARCHAR(255) DEFAULT 'default',    payload TEXT NOT NULL,    status ENUM('pending', 'processing', 'completed', 'failed') DEFAULT 'pending',    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,    available_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,    attempts INT DEFAULT 0);

Step 2: Adding a job to the queue

function addJob(string $queueName, array $data): void {    $pdo = new PDO('mysql:host=localhost;dbname=test', 'user', 'password');    $stmt = $pdo->prepare(        "INSERT INTO jobs (queue_name, payload, available_at)          VALUES (?, ?, NOW())"    );    $stmt->execute([$queueName, json_encode($data)]);}

// Example usageaddJob('emails', ['to' => 'user@example.com', 'subject' => 'Hello!']);

Step 3: Creating a worker (worker.php)

// Infinite loop for processing jobswhile (true) {    $pdo = new PDO('mysql:host=localhost;dbname=test', 'user', 'password');        // Capture a job with row locking    $stmt = $pdo->prepare(        "SELECT * FROM jobs          WHERE status = 'pending' AND available_at <= NOW()          ORDER BY id ASC          LIMIT 1 FOR UPDATE"    );    $stmt->execute();    $job = $stmt->fetch(PDO::FETCH_ASSOC);        if ($job) {        // Mark as "processing"        $updateStmt = $pdo->prepare("UPDATE jobs SET status = 'processing' WHERE id = ?");        $updateStmt->execute([$job['id']]);                try {            // Execute the job            $data = json_decode($job['payload'], true);            processJob($data);                        // Mark as completed            $pdo->prepare("UPDATE jobs SET status = 'completed' WHERE id = ?")                ->execute([$job['id']]);        } catch (Exception $e) {            // Error handling            $attempts = $job['attempts'] + 1;            if ($attempts < 3) {