HTML Parsing in Go (Golang) | PythonLib Library

HTML Parsing in Go (Golang): Library Overview and Practical Examples

Go (Golang) is a compiled, statically typed language that is excellent for building high-performance web scraping and HTML parsing tools. The built-in standard library net/html provides a low-level tokenizer, but for convenient work with the DOM tree and CSS selectors, third-party packages are more commonly used. In this article, we will cover three key tools: goquery, colly, and the standard net/html package. You will learn how to choose the right tool and see working code examples.

1. Standard Library: `net/html`

The golang.org/x/net/html package provides a tokenizer and HTML parser. It implements the HTML5 specification and allows parsing a document into a tree of nodes. This is the lightest and fastest option, but requires writing more code for navigation.

Example: extracting all links (<a href>) from a page:

package main

import (    "fmt"    "golang.org/x/net/html"    "net/http"    "strings")

func main() {    resp, _ := http.Get("https://example.com")    defer resp.Body.Close()

    doc, _ := html.Parse(resp.Body)

    var links []string    var f func(*html.Node)    f = func(n *html.Node) {        if n.Type == html.ElementNode && n.Data == "a" {            for _, attr := range n.Attr {                if attr.Key == "href" {                    links = append(links, attr.Val)                }            }        }        for c := n.FirstChild; c != nil; c = c.NextSibling {            f(c)        }    }    f(doc)

    fmt.Println(strings.Join(links, ""))}

Pros: minimal dependencies, high speed.
Cons: no CSS selectors, need to write recursive traversals.

2. goquery — jQuery for Go

goquery is the most popular library for HTML parsing in Go. It uses net/html under the hood and adds a jQuery-like syntax. You can use CSS selectors to find elements, method chaining, and DOM manipulation.

Installation:

go get github.com/PuerkitoBio/goquery

Example: collecting article titles from a blog:

package main

import (    "fmt"    "github.com/PuerkitoBio/goquery"    "log"    "net/http")

func main() {    resp, err := http.Get("https://habr.com/ru/all/")    if err != nil {        log.Fatal(err)    }    defer resp.Body.Close()

    doc, err := goquery.NewDocumentFromReader(resp.Body)    if err != nil {        log.Fatal(err)    }

    doc.Find("article h2").Each(func(i int, s *goquery.Selection) {        title := s.Text()        link, _ := s.Find("a").Attr("href")        fmt.Printf("%d. %s   %s", i+1, title, link)    })}

Key goquery methods:

Find(selector) — search for elements by CSS selector
Each(func) — iterate over found elements
Attr(name) — get attribute value
Text() — extract text content
Html() — get HTML code inside an element

goquery is ideal for rapid prototyping and parsing pages with complex structure. However, keep in mind: it does not support JavaScript execution — it is a tool for static HTML.