Elasticsearch Python Tutorial: elasticsearch-py for…

Introduction

Elasticsearch is a powerful distributed search engine built on Apache Lucene, designed for full-text search, logging, monitoring, and analytics on large datasets. To integrate it with Python, you use the official client elasticsearch-py, which provides full access to all Elasticsearch APIs through Python code.

The elasticsearch-py library lets developers implement advanced search, aggregations, and large-scale data processing directly in Python applications, including web services, analytics tools, and monitoring systems.

About the elasticsearch-py Library

Elasticsearch-py is the official Python client for Elasticsearch. It provides low-level access to all Elasticsearch features via the RESTful API while offering a convenient Python interface.

Key Features:

Official Support: Developed and maintained by the Elastic team
Full Compatibility: Supports all Elasticsearch versions
Automatic Node Discovery: Can automatically detect and connect to cluster nodes
Connection Pools: Efficient connection management to the cluster
Error Handling: Built-in exception handling system
Serialization: Automatic conversion of Python objects to JSON

Installation and Connecting to a Cluster

Installation

pip install elasticsearch

For working with SSL certificates:

pip install elasticsearch[async]

Connecting to a Local Cluster

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")

Connecting to a Remote Cluster with Authentication

es = Elasticsearch(
    "https://your-cluster:9200",
    basic_auth=("user", "password"),
    verify_certs=True
)

Connecting with API Keys

es = Elasticsearch(
    "https://your-cluster:9200",
    api_key="your_api_key",
    verify_certs=True
)

Connecting to Multiple Nodes

es = Elasticsearch(
    [
        "https://node1:9200",
        "https://node2:9200",
        "https://node3:9200"
    ],
    basic_auth=("user", "password")
)

Core Elasticsearch Concepts

Index — Similar to a table in relational databases, stores documents with a similar structure
Document — A JSON object stored in an index, similar to a row in a table
Field — An attribute of a document, similar to a column in a table
Mapping — Defines the structure of a document and its field types
Shard — A physical unit of data storage for an index
Replica — A copy of a shard for fault tolerance

Managing Indexes

Creating Indexes and Mappings

mapping = {
    "mappings": {
        "properties": {
            "title": {"type": "text", "analyzer": "standard"},
            "tags": {"type": "keyword"},
            "published": {"type": "date"},
            "content": {"type": "text"},
            "author": {"type": "keyword"},
            "views": {"type": "integer"}
        }
    },
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0
    }
}

es.indices.create(index="articles", body=mapping)