PHP: Reading a Gigabyte File Line by Line Without Crashing

Introduction: When Ordinary file_get_contents Kills the Server

Imagine this situation: you are writing a log parser, a CSV importer with millions of rows, or processing a database dump. You write $data = file_get_contents('bigfile.log'); — and your PHP script crashes with the error Fatal error: Allowed memory size exhausted. Sound familiar? If not, you're lucky. But sooner or later, every developer faces the task: "How to read a large file line by line without killing the server?".

On Stack Overflow, the question "How to read a large file line by line?" has garnered hundreds of thousands of views. And for good reason. In this article, we will cover not just one method, but several approaches — from basic to advanced. You will learn how to read files tens of gigabytes in size using only a couple of megabytes of RAM. Let's go!

Why file_get_contents() and file() Are Evil for Large Files

Before diving into optimization, let's understand why the standard functions are unsuitable.

\n
• file_get_contents() — loads the entire file into a string. If the file is 2 GB, PHP will try to allocate 2 GB of RAM. On a hosting plan with a memory_limit of 128 MB, this is a guaranteed fatal error.
\n
• file() — also reads the entire file, but additionally splits it into an array of lines. Memory consumption increases even more (plus the overhead of the array).
\n

The moral: if the file is larger than 10-20% of your memory_limit — forget about these functions. We need a streaming approach.

Method 1: The Classic — fgets() and File Handle

The most basic and reliable method is to open the file with fopen() and read it line by line with fgets(). This approach uses PHP's internal buffer, which is usually 8 KB, and does not load the entire file into memory.

<?php\n$handle = fopen('bigfile.log', 'r');\nif ($handle) {\n    while (($line = fgets($handle)) !== false) {\n        // Process the line\n        echo $line; // or write to DB, or count something\n    }\n    fclose($handle);\n} else {\n    // File open error\n    echo "Failed to open file.";\n}\n?>

Pros: minimal memory consumption, works on any PHP.

Cons: you need to explicitly close the file, no convenient methods for skipping headers or iterating over lines with counting.

This code will handle a file of any size, as long as you don't try to store all lines in an array inside the loop.

Method 2: SplFileObject — Object-Oriented Approach

Since PHP 5.1, we have the excellent SplFileObject class. It implements the Iterator and SeekableIterator interfaces, making it ideal for line-by-line reading.

<?php\n$file = new SplFileObject('bigfile.log');\nwhile (!$file->eof()) {\n    $line = $file->fgets();\n    // Process the line\n    echo $line;\n}\n?>

But there is a nuance! By default, SplFileObject can also load lines into memory if you use current() inside a foreach. To guarantee low memory usage, use fgets() in a while loop, as shown above.

A more advanced option is to set flags for optimization:

<?php\n$file = new SplFileObject('bigfile.log');\n$file->setFlags(SplFileObject::DROP_NEW_LINE); // Remove \\n from the line\n\nforeach ($file as $lineNum => $line) {\n    // $lineNum — line number (starting from 0)\n    echo "Line $lineNum: $line\\n";\n}\n?>

Important: When using foreach with SplFileObject, PHP still reads one line at a time. Memory is not affected. However, if you try to call $file->seek(1000000) and then jump around the file — be careful: seek iterates through