Data Parsing and Scraping in JavaScript: From Beginner to Pro

Why parse data in JavaScript if Python exists?

Hello, friend! If you're reading this text, you're most likely already familiar with Python and its libraries like BeautifulSoup or Scrapy. But what if you encounter a task where Python falls short? For example, you need to collect data from a site that is entirely built on React or Angular. All the information is loaded via JavaScript, and the source HTML is empty. That's where JavaScript comes into play!

Data parsing and scraping in JavaScript is not just a trendy fad, but a pressing necessity for working with modern web applications. Node.js allows you to run a browser engine directly on the server, process the DOM, click buttons, and wait for asynchronous content to load. In this article, we will explore real tools and write code that you can immediately use in your projects.

Ready to turn JavaScript into a powerful data vacuum? Let's go!

Main tools for scraping in JavaScript

Before writing code, let's get acquainted with the main "fighters" in our arsenal. For data parsing in JavaScript, we will use three pillars:

\n
• Axios — an HTTP client for sending requests and receiving page HTML code.
\n
• Cheerio — a fast and lightweight library for parsing HTML that works like jQuery on the server.
\n
• Puppeteer — a full-fledged headless browser (Chrome) that can execute JavaScript, click, scroll, and wait.
\n

For static pages, Axios + Cheerio will suffice. If the site loads data via AJAX or requires authorization, use Puppeteer. Now let's move on to practice.

Example 1: Simple parsing of a static page with Cheerio

Let's say we need to collect news headlines from a site that delivers HTML immediately (without dynamics). Install the dependencies:

npm install axios cheerio

Now let's write a script that extracts all h2 headlines from the main page (replace the URL with a real one):

const axios = require('axios');\nconst cheerio = require('cheerio');\n\nasync function parseStaticPage(url) {\n    try {\n        const { data } = await axios.get(url);\n        const $ = cheerio.load(data);\n\n        const titles = [];\n        $('h2').each((i, el) => {\n            titles.push($(el).text().trim());\n        });\n\n        console.log('Found headlines:');\n        titles.forEach((title, index) => {\n            console.log(`${index + 1}. ${title}`);\n        });\n    } catch (error) {\n        console.error('Parsing error:', error.message);\n    }\n}\n\nparseStaticPage('https://example.com/news');

This code is the foundation to start learning data scraping in JavaScript. Cheerio works lightning fast, but cannot execute scripts. If the site shows "Loading..." — read on.

Example 2: Parsing dynamic pages with Puppeteer

Now imagine that data appears only after the user scrolls the page or clicks the "Show more" button. Puppeteer is ideal for such tasks. Let's install it:

npm install puppeteer

Scenario: go to a page with products, wait 3 seconds (for JS to load), then collect product names:

const puppeteer = require('puppeteer');\n\nasync function parseDynamicPage(url) {\n    const browser = await puppeteer.launch({ headless: true });\n    const page = await browser.newPage();\n\n    await page.goto(url, { waitUntil: 'networkidle2' });\n    // Wait for content to appear (selector .product-title)\n    await page.waitForSelector('.product-title', { timeout: 10000 });\n\n    const items = await page.evaluate(() => {\n        const titles = [];\n        document.querySelectorAll('.product-title').forEach(el => {