Working with S3 and Cloud Storage in Node.js: @aws-sdk/client-s3 Library

Working with S3 and Cloud Storage in JavaScript (Node.js)

S3 (Simple Storage Service) is the de facto standard for object storage in the cloud. Whether you use Amazon S3, Yandex Object Storage, Mail.ru Cloud Storage, or a local solution like MinIO, the API remains the same. In this article, we'll cover how to connect to S3-compatible storage from Node.js using the official AWS SDK v3.

What is S3 and why is it needed?

S3 allows you to store an unlimited number of files (objects) in "buckets". Each object has a unique key (path) and metadata. Main use cases:

• Storing user files (avatars, documents)
• Database and log backups
• Static files for web applications (images, CSS, JS)
• Storing media files (video, audio)

Installing and Configuring AWS SDK v3

Starting from version 3, the AWS SDK has become modular. To work with S3, you only need to install one package:

npm install @aws-sdk/client-s3

You will also need environment variables with access keys. It's best to store them in a .env file:

AWS_ACCESS_KEY_ID=your_access_keyAWS_SECRET_ACCESS_KEY=your_secret_keyAWS_REGION=ru-central1S3_ENDPOINT=https://storage.yandexcloud.netS3_BUCKET=my-bucket

Connecting to an S3-Compatible Storage

Let's create an S3 client with a custom endpoint (for Yandex Object Storage, MinIO, and others):

const { S3Client } = require('@aws-sdk/client-s3');require('dotenv').config();

const s3Client = new S3Client({  region: process.env.AWS_REGION,  endpoint: process.env.S3_ENDPOINT,  credentials: {    accessKeyId: process.env.AWS_ACCESS_KEY_ID,    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,  },  forcePathStyle: true, // important for MinIO and some S3-compatible storage});

module.exports = s3Client;

Basic S3 Operations

1. Uploading a File (PutObject)

Upload a file from a buffer or stream:

const { PutObjectCommand } = require('@aws-sdk/client-s3');const fs = require('fs');

async function uploadFile(filePath, key) {  const fileContent = fs.readFileSync(filePath);    const command = new PutObjectCommand({    Bucket: process.env.S3_BUCKET,    Key: key,    Body: fileContent,    ContentType: 'image/png', // specify the correct MIME type  });

  try {    const response = await s3Client.send(command);    console.log('File uploaded:', response.$metadata.httpStatusCode);    return `https://storage.yandexcloud.net/${process.env.S3_BUCKET}/${key}`;  } catch (err) {    console.error('Upload error:', err);  }}

2. Downloading a File (GetObject)

Get an object and save it locally:

const { GetObjectCommand } = require('@aws-sdk/client-s3');const fs = require('fs');const stream = require('stream');const { promisify } = require('util');const pipeline = promisify(stream.pipeline);

async function downloadFile(key, destinationPath) {  const command = new GetObjectCommand({    Bucket: process.env.S3_BUCKET,    Key: key,  });

  try {    const response = await s3Client.send(command);    const writeStream = fs.createWriteStream(destinationPath);    await pipeline(response.Body, writeStream);    console.log('File downloaded:', destinationPath);  } catch (err) {    console.error('Download error:', err);  }}

3. Listing Objects (ListObjectsV2)

Useful for displaying files in a bucket:

const { ListObjectsV2Command } = require('@aws-sdk/client-s3');

async function listFiles(prefix = '') {  const command = new ListObjectsV2Command({    Bucket: process.env.S3_BUCKET,    Prefix: prefix,    MaxKeys: 100,  });

  try {    const response = await s3