Why Adding Arrays Is Faster in Two Loops Than in One: Analysis of C++ Errors

Why Adding Arrays Is Faster in Two Loops Than in One? Analysis of a Legendary C++ Error

Have you ever written code that intuitively seemed correct but ran slowly? And then, by accidentally splitting one loop into two, you got a 2–3x speedup? A familiar pain. On Stack Overflow, there is a legendary question that has gathered thousands of views and hundreds of comments: «Why are elementwise additions much faster in separate loops than in a combined loop?». Today, we will break down this phenomenon in simple terms, with C++ code examples and without overly complex theories.

This is not magic. This is cache misses and how the processor actually accesses memory. If you write in C++ and want your code to fly — read carefully. At the end of the article, there are specific action items you can apply right now.

The Core Problem: Two Loops vs. One

Imagine we have two large arrays a and b, and we want to perform elementwise addition, followed by multiplying each element. Intuition suggests: «Let's make one loop to avoid bouncing the processor back and forth!».

Here is what the combined loop looks like:

const int N = 1000000;\nfloat a[N], b[N];\n\n// One loop — doing everything at once\nfor (int i = 0; i < N; ++i) {\n    a[i] = a[i] + b[i];  // addition\n    a[i] = a[i] * 2.0f;  // multiplication\n}

And here are two separate loops:

// First loop — only addition\nfor (int i = 0; i < N; ++i) {\n    a[i] = a[i] + b[i];\n}\n\n// Second loop — only multiplication\nfor (int i = 0; i < N; ++i) {\n    a[i] = a[i] * 2.0f;\n}

It's logical to assume the first option is faster: less overhead for loop organization. But in practice, the opposite is true — the second option is often 2–3 times faster. Why? Welcome to the world of cache hierarchy.

Cache Misses: The Main Enemy of Performance

Modern processors do not access RAM directly every time. They use cache — fast memory located directly on the chip. Cache comes in several levels (L1, L2, L3), and each subsequent level is slower but larger.

When your code reads or writes data, the processor loads an entire block (cache line) — typically 64 bytes — into the cache. If the next request hits the same block, it's a hit. If not, it's a miss, and the processor has to wait for data from slow RAM. A single miss can cost hundreds of clock cycles.

Now, let's return to our example. In the combined loop, on each iteration, we access a[i] and b[i] twice: first for reading, then for writing. While we work with a[0] and b[0], the cache pulls in their cache lines. But on the next iteration, we go to a[1] and b[1] — a miss is possible if the data is in different cache lines. As a result, we constantly "jerk" the memory, forcing the cache to reload.

In two separate loops, everything is different. The first loop sequentially reads a[i] and b[i] and writes the result back. The data goes linearly — the cache is predictable and works perfectly (spatial locality). The second loop passes over the same array a again — the data is already hot in the L1 cache. There are practically no misses.

Top Errors That Kill Performance

Error #1 — mixing operations with different arrays in a single loop. If you can split the logic into two passes (especially with large data) — do it. The cache will thank you.

Error #2 — ignoring the cache line size. 64 bytes is roughly 16 floats or 8 doubles. If your arrays have strange strides or you jump through indices non-sequentially, you are guaranteed cache misses.

Error #3