Working with Kafka in PHP: Libraries, Installation, and Code Examples

Working with Kafka in PHP: Library Overview and Practical Examples

Apache Kafka is a distributed streaming platform that has become the de facto standard for building high-load messaging systems. In the PHP ecosystem, integration with Kafka is possible through several libraries, each with its own features. In this article, we will cover two main libraries: php-rdkafka (a C extension) and longlang/phpkafka (pure PHP), and show code examples for a producer and consumer.

1. The php-rdkafka Library

php-rdkafka is a native PHP extension based on the librdkafka library. It provides maximum performance and full control over Kafka configuration. Suitable for high-load production projects.

Installation

Installation requires librdkafka and PECL:

# Install librdkafka (Ubuntu/Debian)sudo apt-get install librdkafka-dev

# Install the extension via PECLsudo pecl install rdkafka

# Enable in php.iniextension=rdkafka.so

Example: Producer

The producer sends messages to a Kafka topic. It is important to properly configure asynchronous sending and error handling:

<?phpuse RdKafka\\Producer;use RdKafka\\Conf;

$conf = new Conf();$conf->set('bootstrap.servers', 'localhost:9092');$conf->set('acks', 'all'); // wait for acknowledgment from all replicas

$producer = new Producer($conf);$topic = $producer->newTopic('test-topic');

// Send message$topic->produce(RD_KAFKA_PARTITION_UA, 0, 'Hello Kafka!');$producer->flush(1000); // wait for delivery (1 sec)

echo "Message sent";

Example: Consumer

The consumer reads messages from a topic. It is recommended to use consumer groups for load balancing:

<?phpuse RdKafka\\KafkaConsumer;use RdKafka\\Conf;

$conf = new Conf();$conf->set('bootstrap.servers', 'localhost:9092');$conf->set('group.id', 'my-consumer-group');$conf->set('auto.offset.reset', 'earliest');

$consumer = new KafkaConsumer($conf);$consumer->subscribe(['test-topic']);

echo "Waiting for messages...";while (true) {    $message = $consumer->consume(120 * 1000); // timeout 2 minutes    switch ($message->err) {        case RD_KAFKA_RESP_ERR_NO_ERROR:            echo "Received: " . $message->payload . "";            break;        case RD_KAFKA_RESP_ERR__PARTITION_EOF:            echo "End of partition reached";            break;        case RD_KAFKA_RESP_ERR__TIMED_OUT:            echo "Timeout";            break;        default:            echo "Error: " . $message->errstr() . "";            break;    }}

2. The longlang/phpkafka Library

longlang/phpkafka is a pure PHP implementation of the Kafka protocol with no external dependencies. It is easier to install (no compilation required) and is suitable for development, testing, or projects with low performance requirements.

Installation

Installation via Composer:

composer require longlang/phpkafka

Example: Producer

<?phpuse longlang\\phpkafka\\Producer\\Producer;use longlang\\phpkafka\\Producer\\ProducerConfig;

$config = new ProducerConfig();$config->setBootstrapServer('localhost:9092');$config->setUpdateBrokers(true);

$producer = new Producer($config);$producer->send('test-topic', 'Hello Kafka from pure PHP!', 'key-1');echo "Message sent";

Example: Consumer

<?phpuse longlang\\phpkafka\\Consumer\\Consumer;use longlang\\phpkafka\\Consumer\\ConsumerConfig;

$config = new ConsumerConfig();$config->setBootstrapServer('localhost:9092');$config->setGroupId('my-group');$config->setTopic('te