Parsing HTML in C# (.NET): HtmlAgilityPack and AngleSharp Libraries

Parsing HTML in C# (.NET): Overview of HtmlAgilityPack and AngleSharp Libraries

HTML parsing is one of the key tasks when developing web scrapers, content analyzers, or automated tests. In the .NET ecosystem, there are several powerful libraries that allow you to extract data from HTML documents, manipulate the DOM tree, and perform complex queries. In this article, we will look at the two most popular libraries: HtmlAgilityPack and AngleSharp. You will learn their features, advantages, and get ready-made code examples in C#.

HtmlAgilityPack: The Classic Approach

HtmlAgilityPack (HAP) is the oldest and most common library for parsing HTML in .NET. It provides an intuitive API for navigating the DOM tree, supports XPath and LINQ queries. Key features:

• Loading HTML from a string, file, or URL.
• Support for XPath 1.0 for precise element searching.
• Working with attributes, text, and nested nodes.
• Automatic correction of malformed HTML (e.g., unclosed tags).

Installation

Install the library via NuGet:

Install-Package HtmlAgilityPack

Or via .NET CLI:

dotnet add package HtmlAgilityPack

Example: Extracting News Headlines

Suppose we need to parse headlines from a news website. Here's how to do it using HAP and XPath:

using HtmlAgilityPack;using System;using System.Linq;

class Program{    static void Main()    {        // Load the HTML document        var web = new HtmlWeb();        var doc = web.Load("https://example-news.com");                // Select all h2 headings with class "news-title"        var titles = doc.DocumentNode            .SelectNodes("//h2[@class='news-title']");                if (titles != null)        {            foreach (var title in titles)            {                Console.WriteLine(title.InnerText.Trim());            }        }    }}

In this example, we use the XPath expression //h2[@class='news-title'] to find all <h2> elements with the specified class. The SelectNodes method returns a collection of nodes, and InnerText extracts the text without HTML tags.

Working with Attributes

Often you need to get the value of an attribute, for example, links from <a> tags:

var links = doc.DocumentNode.SelectNodes("//a[@href]");if (links != null){    foreach (var link in links)    {        string href = link.GetAttributeValue("href", "");        Console.WriteLine(href);    }}

AngleSharp: A Modern Parser with CSS Selectors

AngleSharp is a more modern library that supports not only XPath but also CSS selectors, making it convenient for developers familiar with web technologies. It also fully complies with W3C standards and can work with dynamic content.

• Support for CSS selectors (like in jQuery).
• Working with HTML, XML, SVG, and MathML.
• Ability to execute JavaScript (via extensions).
• Asynchronous document loading.

Installation

dotnet add package AngleSharp

Example: Extracting Data Using CSS Selectors

Let's repeat the previous example, but now with AngleSharp and CSS selectors:

using AngleSharp;using AngleSharp.Html.Parser;using System;using System.Linq;using System.Threading.Tasks;

class Program{    static async Task Main()    {        // Create a parser        var parser = new HtmlParser();                // Load HTML (can be from a string or URL)        var document = await parser.ParseDocumentAsync(            a