Reading Giant Files in PHP Line by Line Without Killing the Server

Introduction: "Fatal Error" and 2 Gigabytes of Logs

Imagine: you are a developer, and a system administrator comes to you with a flash drive. On it is a web server log file for the last six months. The size? A modest 2.5 gigabytes. "I need to find all the IP addresses from which there were brute force attempts," he says and goes off to drink coffee.

You open your IDE and mechanically write:

$lines = file('huge_log.txt');foreach ($lines as $line) {    // parsing...}

You press "Run"... and see a white screen in the browser, and in the error logs — Fatal Error: Allowed memory size exhausted. Sound familiar? This is the classic trap that almost all PHP developers have fallen into.

The question "How to read a large file line by line?" on Stack Overflow has gathered hundreds of thousands of views for a reason. This is one of those tasks where a "naive" solution kills the server, and the right approach saves the project. Today we will break down how to read huge files in PHP without exceeding the memory limit and while maintaining high performance. Let's go!

Why file() and file_get_contents() Are Evil for Large Data

Let's dispel the main myth right away. The file() and file_get_contents() functions are greedy functions. They load the entire file into RAM at once. If your file weighs 2 GB, and the memory_limit in php.ini is set to 128 MB (a standard value for many hosting providers), you are guaranteed to get an error.

Moreover, PHP stores strings in an encoding that can increase memory consumption by 2-4 times relative to the file size on disk. That means 2 GB of logs can "eat up" 6-8 GB of RAM!

Conclusion: if the file size exceeds 10-20% of your memory_limit, forget about file(). We need a streaming approach.

The Right Way: fgets() and Streaming Reading

The most reliable and performant method is to use fopen() paired with fgets(). These functions do not read the entire file, but one line at a time. Imagine you are drinking juice through a straw — you don't drink the whole glass in one gulp, but sip it little by little. Streaming reading works the same way.

Here is a basic, but already working example:

<?php$handle = fopen('huge_log.txt', 'r');if ($handle) {    while (($line = fgets($handle)) !== false) {        // Process one line        echo $line;    }    fclose($handle);}?>

What is happening here? fopen() opens the file and returns a pointer (resource). fgets() reads one line at a time up to the newline character (). Memory is allocated only for the current line, not for the entire file. After processing, the line is destroyed by the garbage collector (or overwritten).

Important Performance Nuances

• Buffer size: fgets() reads up to 8192 bytes at a time by default. If lines are longer, the function will make several iterations. For the vast majority of logs, this is sufficient.
• Checking for false: Always use strict comparison !== false. Why? Because a line can be empty ("") or contain "0", and while ($line = fgets($handle)) might stop prematurely.
• Closing the file: Always call fclose(). Yes, PHP will do this itself when the script finishes, but if you are processing a file in a loop or in a background script, unclosed descriptors are a resource leak.

Supercharge: Generators (yield) for Reusability

Manual management of fopen/fclose is good, but if you need to read different files in different parts of the project, the code will start to duplicate. This is where generators come to the rescue — one of the most elegant features of PHP.

A generator allows you to create a function that