Working with S3 and Cloud Storage in Java: AWS SDK Library

Working with S3 and Cloud Storage in Java using AWS SDK

Cloud object storage using the S3 protocol (Amazon Simple Storage Service) has become the de facto standard for storing unstructured data. In Java, the official AWS SDK for Java library is used to work with S3-compatible storage (Amazon S3, MinIO, Yandex Object Storage, DigitalOcean Spaces, etc.). In this article, we will cover setup, client initialization, and basic file operations.

Installing AWS SDK

To work with S3, add the dependency to pom.xml (Maven):

<dependency>    <groupId>software.amazon.awssdk</groupId>    <artifactId>s3</artifactId>    <version>2.27.0</version></dependency>

For Gradle:

implementation 'software.amazon.awssdk:s3:2.27.0'

Creating an S3 Client

The client is configured via a builder. For Amazon S3, simply specify the region:

S3Client s3 = S3Client.builder()    .region(Region.EU_CENTRAL_1)    .build();

For compatible storage (MinIO, Yandex Object Storage), you need to specify the endpoint and credentials:

String endpoint = "https://storage.yandexcloud.net";String accessKey = "YOUR_ACCESS_KEY";String secretKey = "YOUR_SECRET_KEY";

S3Client s3 = S3Client.builder()    .endpointOverride(URI.create(endpoint))    .region(Region.US_EAST_1) // region can be any    .credentialsProvider(StaticCredentialsProvider.create(        AwsBasicCredentials.create(accessKey, secretKey)))    .build();

Basic Bucket Operations

• Creating a bucket
• Checking existence
• Deleting a bucket

Example of creating a bucket:

String bucketName = "my-java-bucket";try {    s3.createBucket(CreateBucketRequest.builder()        .bucket(bucketName)        .build());    System.out.println("Bucket created: " + bucketName);} catch (S3Exception e) {    System.err.println("Error: " + e.awsErrorDetails().errorMessage());}

Uploading Files (Upload)

You can upload a file from a stream, string, or local file:

// Upload from local files3.putObject(PutObjectRequest.builder()        .bucket(bucketName)        .key("docs/report.pdf")        .build(),    RequestBody.fromFile(Paths.get("/home/user/report.pdf")));

// Upload string as a text files3.putObject(PutObjectRequest.builder()        .bucket(bucketName)        .key("hello.txt")        .contentType("text/plain")        .build(),    RequestBody.fromString("Hello, S3!"));

Downloading Files (Download)

To retrieve an object, use getObject:

// Download to files3.getObject(GetObjectRequest.builder()        .bucket(bucketName)        .key("docs/report.pdf")        .build(),    ResponseTransformer.toFile(Paths.get("downloaded_report.pdf")));

// Download to stringString content = s3.getObjectAsBytes(GetObjectRequest.builder()        .bucket(bucketName)        .key("hello.txt")        .build())    .asUtf8String();System.out.println(content);

Deleting Objects

s3.deleteObject(DeleteObjectRequest.builder()    .bucket(bucketName)    .key("hello.txt")    .build());System.out.println("File deleted");

Listing Objects in a Bucket

To get a list of all files (with pagination):

ListObjectsV2Request listReq = ListObjectsV2Request.builder()    .bucket(bucketName)    .maxKeys(100)    .build();

ListObjectsV2Response listRes = s3.listObjectsV2(listReq);for (S3Object obj : listRes.contents()) {    System.out.println(" - "