Working with S3 and Cloud Storage in Kotlin: AWS SDK and Ktor S3 Client

Working with S3 and Cloud Storage in Kotlin

In modern Kotlin development, there is often a need to integrate with cloud storage compatible with the Amazon S3 (Simple Storage Service) protocol. This can include AWS S3, MinIO, DigitalOcean Spaces, Yandex Object Storage, and others. In this article, we will look at two main approaches: using the official AWS SDK for Kotlin and a lightweight client based on Ktor.

1. AWS SDK for Kotlin (aws-sdk-kotlin)

The official SDK from Amazon provides a typed and asynchronous API that fully supports Kotlin coroutines. This is the most comprehensive solution for working with S3.

Adding the Dependency

Add to build.gradle.kts:

dependencies {    implementation("aws.sdk.kotlin:s3:1.0.0")    implementation("aws.smithy.kotlin:http-client-engine-ktor:1.0.0")}

Initializing the Client

import aws.sdk.kotlin.services.s3.S3Clientimport aws.sdk.kotlin.services.s3.model.*

suspend fun createClient(): S3Client = S3Client.fromEnvironment {    region = "ru-central1"    // For MinIO or other S3-compatible storage:    endpointUrl = "https://storage.yandexcloud.net"    // Configure via environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY}

Uploading a File (PutObject)

suspend fun uploadFile(client: S3Client, bucket: String, key: String, filePath: String) {    val fileBytes = java.io.File(filePath).readBytes()    client.putObject(        PutObjectRequest {            this.bucket = bucket            this.key = key            this.body = fileBytes.toByteArray()        }    )    println("File $key uploaded to bucket $bucket")}

Downloading a File (GetObject)

suspend fun downloadFile(client: S3Client, bucket: String, key: String, savePath: String) {    val response = client.getObject(        GetObjectRequest {            this.bucket = bucket            this.key = key        }    )    val bytes = response.body?.readAll()    if (bytes != null) {        java.io.File(savePath).writeBytes(bytes)        println("File $key saved to $savePath")    }}

Listing Objects in a Bucket

suspend fun listObjects(client: S3Client, bucket: String): List<String> {    val response = client.listObjectsV2(        ListObjectsV2Request {            this.bucket = bucket        }    )    return response.contents?.map { it.key ?: "" } ?: emptyList()}

2. Ktor S3 Client (Lightweight Alternative)

If you don't need the full power of the AWS SDK, you can use Ktor with the ktor-client-s3 library. This provides minimal weight and full control over HTTP requests.

Adding Dependencies

dependencies {    implementation("io.ktor:ktor-client-core:2.3.0")    implementation("io.ktor:ktor-client-cio:2.3.0")    implementation("io.ktor:ktor-client-content-negotiation:2.3.0")    implementation("io.ktor:ktor-serialization-kotlinx-json:2.3.0")    // For signing S3 requests    implementation("io.ktor:ktor-client-auth:2.3.0")}

Creating an S3 Client with Ktor

import io.ktor.client.*import io.ktor.client.request.*import io.ktor.client.statement.*import io.ktor.http.*import io.ktor.utils.io.*

class S3KtorClient(    private val endpoint: String,    private val accessKey: String,    private val secretKey: String) {    private val client = HttpClient {        install(io.ktor.client.plugins.auth.Auth) {            basic {                credentials {                    username = accessKey                    password = secretKey                }            }        }    }

    suspend fun putObject(bucket: String, key: String, data: ByteArray) {        client.put("