Parsing HTML in Go (Golang)

Parsing HTML in Go (Golang): Tools Overview and Examples

Go (Golang) is a compiled, statically typed programming language from Google that is ideal for creating high-performance parsers and web scrapers. Thanks to built-in concurrency support (goroutines) and a rich standard library, Go allows you to efficiently process HTML documents of any complexity. In this article, we will look at the main libraries for parsing HTML in Go, their capabilities, and provide practical code examples.

Standard Library: `net/html`

The golang.org/x/net/html package is the foundation for working with HTML in Go. It provides a tokenizer and parser that builds a DOM tree of the document. This is a low-level tool that gives full control over the parsing process but requires more code for typical tasks.

Example of using net/html to extract all links from an HTML document:

package main

import (    "fmt"    "golang.org/x/net/html"    "strings")

func main() {    htmlContent := `<html><body><a href="/page1">Link 1</a><a href="/page2">Link 2</a></body></html>`    doc, _ := html.Parse(strings.NewReader(htmlContent))    var links []string    var f func(*html.Node)    f = func(n *html.Node) {        if n.Type == html.ElementNode && n.Data == "a" {            for _, attr := range n.Attr {                if attr.Key == "href" {                    links = append(links, attr.Val)                }            }        }        for c := n.FirstChild; c != nil; c = c.NextSibling {            f(c)        }    }    f(doc)    for _, link := range links {        fmt.Println(link)    }}

This code recursively traverses the DOM tree, finds all <a> tags, and extracts the value of the href attribute. The method is simple and reliable, but for more complex tasks (e.g., searching by CSS selectors), it is better to use specialized libraries.

The `goquery` Library: jQuery for Go

github.com/PuerkitoBio/goquery is the most popular library for parsing HTML in Go. It provides an API reminiscent of jQuery: selecting elements by CSS selectors, method chaining, DOM manipulation. This makes the code concise and readable.

Installation:

go get github.com/PuerkitoBio/goquery

Example of parsing a page with extraction of headings and links:

package main

import (    "fmt"    "log"    "net/http"

    "github.com/PuerkitoBio/goquery")

func main() {    // Load the HTML page    res, err := http.Get("https://example.com")    if err != nil {        log.Fatal(err)    }    defer res.Body.Close()

    // Parse the HTML    doc, err := goquery.NewDocumentFromReader(res.Body)    if err != nil {        log.Fatal(err)    }

    // Extract h2 headings    doc.Find("h2").Each(func(i int, s *goquery.Selection) {        fmt.Printf("Heading %d: %s", i+1, s.Text())    })

    // Extract all links    doc.Find("a").Each(func(i int, s *goquery.Selection) {        link, _ := s.Attr("href")        text := s.Text()        fmt.Printf("Link %d: %s (%s)", i+1, text, link)    })}

Key features of goquery:

CSS selectors: support for all standard selectors (classes, IDs, attributes, pseudo-classes).
Method chaining: Find().Filter().Each() — code is written like in jQuery.
Data extraction: methods Text(), Html(), Attr().
DOM modification: SetText(), SetHtml(), Append().