Reading a File Line by Line in Go: Best Practices 2025

Reading a File Line by Line in Go: From Beginner to Pro in 2025

Have you ever wondered how to efficiently read huge log files or process CSV data in Go? If so, you've come to the right place. The topic of reading a file line by line is one of the most popular on Stack Overflow, and for good reason. Go gives us powerful tools, but choosing the right approach can sometimes be tricky.

In this article, we won't just show you how to read lines from a file. We'll break down the best practices for 2025: from the fast bufio.Scanner to custom buffered solutions. You'll learn how to avoid pitfalls (e.g., lines longer than 64KB), handle errors correctly, and speed up work with large data.

Ready? Let's go!

Why Reading Lines Isn't Just "Take and Read"?

In Go, there are several ways to read files, each for its own task. Some use ioutil.ReadFile (by the way, it's deprecated since Go 1.16), others use os.File with a buffer. But for line-by-line processing, the king is bufio.Scanner.

Why? Because it:

\n
• Is memory efficient — doesn't load the entire file into RAM
\n
• Is easy to use — minimal code
\n
• Handles errors — checks every step
\n

But there's a nuance: the default Scanner buffer is 64 KB. If your line is longer, you'll get an error. For such cases, other approaches are needed.

Basic Example: bufio.Scanner

Let's start with the most common method. Here's the minimal working code:

package main\n\nimport (\n    "bufio"\n    "fmt"\n    "log"\n    "os"\n)\n\nfunc main() {\n    file, err := os.Open("data.txt")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer file.Close()\n\n    scanner := bufio.NewScanner(file)\n    for scanner.Scan() {\n        line := scanner.Text()\n        fmt.Println(line)\n    }\n\n    if err := scanner.Err(); err != nil {\n        log.Fatal(err)\n    }\n}\n

What's important here?

\n
• defer file.Close() — closes the file even on panic
\n
• scanner.Text() — returns the line without the newline character (\\n)
\n
• scanner.Err() — checks if there were any errors during reading
\n

This code is ideal for 90% of tasks: configs, logs, small CSVs.

The Problem of Long Lines: Increasing the Buffer

Imagine: you're reading a file with JSON lines of 100 KB each. The Scanner will fail with the error bufio.Scanner: token too long. The solution is to increase the buffer:

scanner := bufio.NewScanner(file)\nbuf := make([]byte, 0, 1024*1024) // 1 MB\nscanner.Buffer(buf, 10*1024*1024) // max 10 MB per line\n\nfor scanner.Scan() {\n    // ... processing\n}\n

Important: the second parameter is the maximum line size. If the line is larger, you'll get an error again. Choose with a margin, but not a huge one — memory isn't infinite.

When Scanner Isn't Suitable: Alternatives

Despite its popularity, bufio.Scanner isn't universal. Here are two cases where you should consider a different approach:

1. Huge Files (Gigabytes)

If the file is larger than available memory (rare, but it happens), Scanner is still good. But if you need to read binary data or process lines with custom delimiters, it's better to use bufio.Reader:

reader := bufio.NewReader(file)\nfor {\n    line, err := reader.ReadString('\\n')\n    if err != nil {\n        break\n    }\n    // line contains \\n at the end\n    line = strings.TrimRight(line, "\\n")\n    fmt.Println(line)\n}\n

Advantage: you control every read operation. Disadvantage: more code and higher risk of errors.

2. Processing Without Loading into Memory

Sometimes you just need to count lines or find a pattern. For this, bufio.Scanner is suitable