Working with S3 and Cloud Storage in Rust

Working with S3 and Cloud Storage in Rust: aws-sdk-s3 library

Cloud object storage services such as Amazon S3 have become the standard for storing and delivering files, backups, and media content. Rust, as a systems programming language, offers powerful and safe tools for integrating with these services. The main library is aws-sdk-s3 — the official SDK from Amazon for Rust.

In this article, we will cover how to set up a project, perform basic operations (upload, download, delete), and work with S3-compatible storage (e.g., MinIO or Yandex Object Storage).

Installation and Setup

Add to Cargo.toml:

[dependencies]aws-sdk-s3 = "1.0"aws-config = "1.0"tokio = { version = "1", features = ["full"] }

For authentication, use environment variables or the ~/.aws/credentials file:

export AWS_ACCESS_KEY_ID="your_access_key"export AWS_SECRET_ACCESS_KEY="your_secret_key"export AWS_DEFAULT_REGION="us-east-1"

Client Initialization

Let's create an async S3 client:

use aws_sdk_s3::Client;use aws_config::load_defaults;use aws_config::BehaviorVersion;

#[tokio::main]async fn main() {    let config = load_defaults(BehaviorVersion::latest()).await;    let client = Client::new(&config);    // now client is ready to use}

Uploading a File to S3 (Upload)

To upload data, use the put_object method. Data can be passed as bytes or from a stream:

use aws_sdk_s3::primitives::ByteStream;use std::path::Path;

async fn upload_file(client: &Client, bucket: &str, key: &str, file_path: &str) -> Result<(), Box<dyn std::error::Error>> {    let body = ByteStream::from_path(Path::new(file_path)).await?;        client        .put_object()        .bucket(bucket)        .key(key)        .body(body)        .send()        .await?;        println!("File {} uploaded to {}/{}", file_path, bucket, key);    Ok(())}

Downloading a File (Get Object)

To retrieve an object and save it locally:

use tokio::io::AsyncWriteExt;

async fn download_file(client: &Client, bucket: &str, key: &str, dest_path: &str) -> Result<(), Box<dyn std::error::Error>> {    let response = client        .get_object()        .bucket(bucket)        .key(key)        .send()        .await?;        let mut file = tokio::fs::File::create(dest_path).await?;    let mut stream = response.body;        while let Some(bytes) = stream.try_next().await? {        file.write_all(&bytes).await?;    }        println!("File {}/{} downloaded to {}", bucket, key, dest_path);    Ok(())}

Listing Objects in a Bucket

Get a list of files (keys) in a bucket:

async fn list_objects(client: &Client, bucket: &str) -> Result<(), Box<dyn std::error::Error>> {    let objects = client        .list_objects_v2()        .bucket(bucket)        .send()        .await?;        if let Some(contents) = objects.contents {        for obj in contents {            if let Some(key) = obj.key {                println!("- {}", key);            }        }    }    Ok(())}

Deleting an Object

async fn delete_object(client: &Client, bucket: &str, key: &str) -> Result<(), Box<dyn std::error::Error>> {    client        .delete_object()        .bucket(bucket)        .key(key)        .send()        .await?;        println!("Object {}/{} deleted", bucket, key);    Ok(())}