JavaScript vs Gigabytes: How the Browser Digests Big Data

When the Browser is Not a Toy, but a Data Center

We are used to JavaScript living in the browser sandbox: animations, clicks, forms. But what if I told you that with pure JS you can process a 500 MB CSV file, build a chart with 100,000 points, and not crash the tab? This is not science fiction, but the proper use of processing big data with JavaScript.

In this article, I will show you how to work with large volumes of data directly in the browser — without a backend, without Python, without copying gigabytes to the server. You will learn about streaming, Web Workers, memory optimization, and libraries that turn the browser into a mini data processor. Let's go!

Why "Naive" Data Processing Kills Performance

Imagine: you load a 300 MB CSV file and write:

const data = await response.json(); // ❌ Fatal error

What happens? JS tries to parse the entire file into RAM. If the user has 4 GB of RAM — the browser will crash. Even on a powerful PC, the garbage collector will start to "stutter", and the interface will freeze for a minute.

Rule #1: don't load everything at once. Big data in JavaScript requires streaming and chunking.

Data Streaming: Reading a File Piece by Piece

The browser API FileReader allows you to read a file in parts. But even more convenient is ReadableStream. Example: processing a large JSON file line by line.

Example: Streaming JSON with NDJSON

The NDJSON format (each line is a separate JSON object) is ideal for big data. Here's how to read it as a stream:

async function* streamJSON(file) {\n  const stream = file.stream();\n  const reader = stream.getReader();\n  const decoder = new TextDecoder();\n  let buffer = '';\n\n  while (true) {\n    const { done, value } = await reader.read();\n    if (done) break;\n    buffer += decoder.decode(value, { stream: true });\n    const lines = buffer.split('\\n');\n    buffer = lines.pop(); // remainder in buffer\n    for (const line of lines) {\n      if (line.trim()) yield JSON.parse(line);\n    }\n  }\n}\n\n// Usage\nfor await (const obj of streamJSON(selectedFile)) {\n  processRow(obj); // process one object at a time\n}\n

Memory barely grows — we only keep one line in it. This approach is the foundation of big data processing with JavaScript in real projects.

Web Workers: Don't Freeze the Interface

Even streaming runs in the main thread. If calculations are complex — the UI will freeze. Solution: Web Workers. This is a separate thread where you can process gigabytes without lag.

Creating a Worker for CSV Processing

Worker code (worker.js):

self.onmessage = async function(e) {\n  const { file, chunkSize } = e.data;\n  const reader = new FileReaderSync(); // synchronous reading — allowed in worker\n  let offset = 0;\n  const results = [];\n\n  while (offset < file.size) {\n    const blob = file.slice(offset, offset + chunkSize);\n    const text = reader.readAsText(blob);\n    const lines = text.split('\\n');\n    for (const line of lines) {\n      if (line.trim()) results.push(parseCSVLine(line));\n    }\n    offset += chunkSize;\n    // Send intermediate result\n    self.postMessage({ type: 'chunk', data: results.splice(0, results.length) });\n  }\n  self.postMessage({ type: 'done' });\n};\n\nfunction parseCSVLine(line) {\n  const [name, age, city] = line.split(',');\n  return { name, age: parseInt(age), city };\n}\n

Main thread:

const worker = new Worker('worker.js');\nworker.postMessage({ file: csvFile, chunkSize: 1024 * 1024 }); // 1 MB chunks\n\nworker.onmessage = (e) => {\n  if (e.data.type === 'chunk') {\n    updateUI(e.data.data); // update table/chart smoothly\n  }\n};\n

Now the interface remains responsive, even if we are processing 2 GB of data. This is a key technique for b