Chaos Engineering for Rust: How to Make Your Application Unkillable

Is Your Rust Server Perfect? Let's Check

You've written a Rust server that processes thousands of requests per second. The compiler is happy, clippy is silent, tests pass. But what happens if the database suddenly goes down? Or the network card starts glitching? Or the disk fills up?

In traditional testing, we check that the code works as intended. Chaos engineering is about something else: we check that the system survives against all odds. And Rust, with its emphasis on reliability and performance, is the perfect playground for such experiments.

This article is not a boring retelling of documentation. We'll go from theory to practice: I'll show you how to implement chaos engineering for Rust applications using real tools and code.

What is Chaos Engineering and Why Does a Rust Developer Need It?

Chaos engineering is a discipline of experimenting on a distributed system to uncover its weaknesses before they manifest in production. Netflix, considered a pioneer of this approach, uses the Chaos Monkey tool to randomly shut down servers.

Why is this especially important for Rust? Because Rust is often chosen for critical systems:

• Network services (proxies, load balancers, API gateways)
• Embedded systems (IoT, automobiles)
• Blockchain nodes and financial systems
• Streaming data processing

In all these cases, the cost of failure is high. Chaos engineering helps ensure that your Rust application will survive not only normal load but also the chaos of the real world.

Tools for Chaos Engineering in the Rust Ecosystem

Although the Rust ecosystem is younger than Java or Go, we already have several mature libraries for injecting chaos.

We'll focus on a practical example using fault-injection and network problem emulation.

Practical Example: "Put Out the Fire" in a Rust Service

Let's say we have a simple HTTP server on axum that does something (e.g., calls an external API). We want to test how it behaves if the external service becomes unavailable or starts responding with a delay.

Step 1: Basic Server with a "Hole"

Let's write a server that makes a request to httpbin.org (simulating an external service):

use axum::{Router, routing::get, response::Json};use reqwest::Client;use std::sync::Arc;use serde_json::Value;

#[tokio::main]async fn main() {    let client = Arc::new(Client::new());

    let app = Router::new()        .route("/fetch", get(handler))        .with_state(client);

    axum::Server::bind(&"0.0.0.0:3000".parse().unwrap())        .serve(app.into_make_service())        .await        .unwrap();}

async fn handler(client: Arc<Client>) -> Result<Json<Value>, (StatusCode, String)> {    let resp = client.get("https://httpbin.org/uuid")        .timeout(std::time::Duration::from_secs(5))        .send()        .await        .map_err(|e| (StatusCode::SERVICE_UNAVAILABLE, e.to_stri