How to Use Object Storage for Application File Uploads

Storing user-uploaded files directly on your application server's disk creates scaling and reliability problems — disk fills up, files are lost if the server fails, and multi-server deployments can't share local storage. Object storage solves this.

Why Move Uploads to Object Storage

• Decouples storage from your application server — scale each independently
• Files survive even if the application server is rebuilt or fails
• Multiple application servers can share the same storage naturally
• Built-in durability from the storage provider, without managing RAID or replication yourself

Prerequisites

• An S3-compatible object storage bucket (self-hosted MinIO or a third-party provider)
• Access credentials for the bucket

Uploading Files from Node.js

const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');

const s3 = new S3Client({
  endpoint: 'https://storage.yourdomain.com',
  region: 'us-east-1',
  credentials: {
    accessKeyId: process.env.S3_ACCESS_KEY,
    secretAccessKey: process.env.S3_SECRET_KEY,
  },
  forcePathStyle: true,
});

async function uploadFile(buffer, key) {
  await s3.send(new PutObjectCommand({
    Bucket: 'myapp-uploads',
    Key: key,
    Body: buffer,
  }));
}

Uploading Files from PHP

use Aws\S3\S3Client;

$s3 = new S3Client([
    'endpoint' => 'https://storage.yourdomain.com',
    'region' => 'us-east-1',
    'credentials' => [
        'key' => getenv('S3_ACCESS_KEY'),
        'secret' => getenv('S3_SECRET_KEY'),
    ],
    'use_path_style_endpoint' => true,
]);

$s3->putObject([
    'Bucket' => 'myapp-uploads',
    'Key' => 'uploads/photo.jpg',
    'SourceFile' => '/tmp/photo.jpg',
]);

Uploading Files from Python

import boto3

s3 = boto3.client(
    's3',
    endpoint_url='https://storage.yourdomain.com',
    aws_access_key_id=os.environ['S3_ACCESS_KEY'],
    aws_secret_access_key=os.environ['S3_SECRET_KEY'],
)

s3.upload_file('local-file.jpg', 'myapp-uploads', 'uploads/photo.jpg')

Generating Pre-Signed URLs (Secure Temporary Access)

Rather than making a bucket public, generate time-limited URLs granting temporary access to a specific file — useful for private user uploads that shouldn't be permanently public:

const { GetObjectCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');

const url = await getSignedUrl(s3, new GetObjectCommand({
  Bucket: 'myapp-uploads',
  Key: 'uploads/private-document.pdf',
}), { expiresIn: 3600 });

Direct Browser-to-Storage Uploads (Bypassing Your Server)

For large files, generate a pre-signed upload URL and have the browser upload directly to object storage, avoiding routing large file transfers through your application server entirely:

const { PutObjectCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');

const uploadUrl = await getSignedUrl(s3, new PutObjectCommand({
  Bucket: 'myapp-uploads',
  Key: `uploads/${filename}`,
}), { expiresIn: 300 });

// Send uploadUrl to the browser, which PUTs the file directly to it

Serving Uploaded Files

Either serve directly from object storage (if the bucket/objects are public), or proxy through your application for access-controlled files, checking authorization before generating a pre-signed URL or streaming the file.

Migrating Existing Local Uploads to Object Storage

aws s3 sync /var/www/myapp/uploads/ s3://myapp-uploads/ --profile minio --endpoint-url https://storage.yourdomain.com

Common Errors

"SignatureDoesNotMatch" — usually a clock synchronization issue between your application server and the storage endpoint, or incorrect credentials; verify NTP is enabled (see How to Set the Correct Timezone and Enable NTP on a Linux VPS).

Uploads fail for large files — check your web server's client_max_body_size (Nginx) if routing through your application, or use direct browser-to-storage uploads to bypass this limit entirely.

Best Practices

• Never hardcode storage credentials in application code — use environment variables
• Use pre-signed URLs for private content rather than making buckets fully public
• Consider direct browser uploads for large files to reduce load on your application server

Continue Reading

• How to Set Up Self-Hosted S3-Compatible Object Storage with MinIO
• How to Manage Environment Variables and Secrets on a VPS
• How to Back Up to Object Storage (S3-Compatible)

Browse more articles in Object Storage, Messaging & APIs.