Watchdog: Real-Time File Monitoring in Python

Introduction to File System Monitoring

In modern software development, monitoring file system changes is a critical task. Tracking file creation, modification, deletion, and movement is essential in various scenarios—from automatically running tests when code changes to monitoring logs in production.

The Watchdog library is a cross-platform Python solution that provides efficient and reliable file system monitoring with minimal overhead. It leverages native operating system mechanisms and offers a unified API for handling file system events.

What is Watchdog

Watchdog is a Python library for real-time file system monitoring. It ensures cross-platform compatibility by using different system-level observation mechanisms depending on the operating system: inotify on Linux, FSEvents on macOS, and ReadDirectoryChangesW on Windows.

Key Benefits

The library offers several important advantages over custom-built solutions or alternative tools. First, it delivers high performance by using native system APIs instead of periodically polling the file system. Second, Watchdog provides a consistent interface across all supported platforms, simplifying cross-platform application development.

Installation and Setup

Install the library using the standard pip command:

pip install watchdog

For additional features, including the watchmedo command-line utility, install the extended version:

pip install watchdog[watchmedo]

The library requires no extra configuration and is ready to use right after installation.

Architecture and Core Components

Watchdog is built around several key components that work together to enable file system monitoring.

Observer

The Observer is the central component of the monitoring system. It starts and manages the observation process, running in a separate thread to keep the main program non-blocking. The Observer can monitor multiple paths simultaneously with different event handlers.

EventHandler

The EventHandler defines how your program should respond to different file system events. Developers create custom handlers by subclassing the base FileSystemEventHandler and overriding the desired methods.

Event

Events are objects containing information about changes in the file system. Each event includes the file or directory path, the operation type, and additional metadata.

Basic Usage Examples

Simple Monitoring

Here's a basic example of setting up a monitoring system:

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class BasicHandler(FileSystemEventHandler):
    def on_modified(self, event):
        if not event.is_directory:
            print(f"File modified: {event.src_path}")
    
    def on_created(self, event):
        if not event.is_directory:
            print(f"File created: {event.src_path}")

observer = Observer()
observer.schedule(BasicHandler(), path="./monitored_folder", recursive=True)
observer.start()

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()
    
observer.join()

Monitoring with Logging

For more detailed event tracking, you can use built-in logging: