Parsing HTML in PHP: DOMDocument, SimpleXML, and cURL for Web Scraping

Parsing HTML in PHP: A Complete Guide to Extracting Data from Web Pages

Parsing HTML is one of the key tasks in web development, allowing you to automatically extract structured data from web pages. PHP offers powerful built-in tools for this purpose, including the DOMDocument and SimpleXML extensions, as well as the cURL library for working with HTTP requests. In this article, we will thoroughly examine how to parse HTML in PHP, covering best practices, error handling, and code examples.

Main Tools for Parsing HTML in PHP

The following main components are used for parsing HTML in PHP:

• DOMDocument — a built-in class for working with the HTML/XML DOM tree. It allows you to load, modify, and extract elements.
• DOMXPath — an extension for DOMDocument that allows executing XPath queries for precise node searching.
• SimpleXML — a simplified extension for working with XML, but it can also be used for HTML if the structure is correct.
• cURL — a library for making HTTP requests, necessary for fetching the HTML code of remote pages.

Step-by-Step Example of Parsing HTML with DOMDocument

Let's look at a complete example of parsing an HTML page. Suppose we need to extract article headings (<h2> tags) and links (<a>) from a website.

<?php// 1. Get the HTML code of the page using cURL$url = 'https://example.com/articles';$ch = curl_init($url);curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36');$html = curl_exec($ch);if (curl_errno($ch)) {    die('cURL error: ' . curl_error($ch));}curl_close($ch);

// 2. Create a DOMDocument object and load the HTML$dom = new DOMDocument();libxml_use_internal_errors(true); // Suppress parsing errors for invalid HTML$dom->loadHTML($html);libxml_clear_errors();

// 3. Create a DOMXPath object for convenient searching$xpath = new DOMXPath($dom);

// 4. Extract all h2 headings$headings = $xpath->query('//h2');echo "<h3>Article headings:</h3>";foreach ($headings as $heading) {    echo "<p>" . htmlspecialchars($heading->nodeValue) . "</p>";}

// 5. Extract all links with class 'article-link'$links = $xpath->query('//a[@class="article-link"]');echo "<h3>Links to articles:</h3>";foreach ($links as $link) {    $href = $link->getAttribute('href');    $text = $link->nodeValue;    echo "<p><a href='" . htmlspecialchars($href) . "'>" . htmlspecialchars($text) . "</a></p>";}?>

Using SimpleXML for Simple Cases

SimpleXML is convenient for parsing well-structured HTML documents but cannot handle invalid HTML. Example of extracting data from a table:

<?php$html = '<table><tr><td>Cell 1</td><td>Cell 2</td></tr></table>';$xml = simplexml_load_string($html);if ($xml === false) {    echo "Error loading HTML";} else {    foreach ($xml->tr->td as $cell) {        echo "Data: " . htmlspecialchars($cell) . "<br>";    }}?>

Error Handling and Best Practices

When parsing HTML in PHP, it is important to consider the following points:

• Suppressing errors: Use libxml_use_internal_errors(true) before loading HTML to avoid displaying warnings about invalid markup.
• Encoding: Ensure the HTML is loaded in the correct encoding (UTF-8). If necessary, use mb_convert_encoding().
• User-Agent: Always