Storing user-uploaded files directly on your application server's disk creates scaling and reliability problems — disk fills up, files are lost if the server fails, and multi-server deployments can't share local storage. Object storage solves this.
Why Move Uploads to Object Storage
- Decouples storage from your application server — scale each independently
- Files survive even if the application server is rebuilt or fails
- Multiple application servers can share the same storage naturally
- Built-in durability from the storage provider, without managing RAID or replication yourself
Prerequisites
- An S3-compatible object storage bucket (self-hosted MinIO or a third-party provider)
- Access credentials for the bucket
Uploading Files from Node.js
const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');
const s3 = new S3Client({
endpoint: 'https://storage.yourdomain.com',
region: 'us-east-1',
credentials: {
accessKeyId: process.env.S3_ACCESS_KEY,
secretAccessKey: process.env.S3_SECRET_KEY,
},
forcePathStyle: true,
});
async function uploadFile(buffer, key) {
await s3.send(new PutObjectCommand({
Bucket: 'myapp-uploads',
Key: key,
Body: buffer,
}));
}
Uploading Files from PHP
use Aws\S3\S3Client;
$s3 = new S3Client([
'endpoint' => 'https://storage.yourdomain.com',
'region' => 'us-east-1',
'credentials' => [
'key' => getenv('S3_ACCESS_KEY'),
'secret' => getenv('S3_SECRET_KEY'),
],
'use_path_style_endpoint' => true,
]);
$s3->putObject([
'Bucket' => 'myapp-uploads',
'Key' => 'uploads/photo.jpg',
'SourceFile' => '/tmp/photo.jpg',
]);
Uploading Files from Python
import boto3
s3 = boto3.client(
's3',
endpoint_url='https://storage.yourdomain.com',
aws_access_key_id=os.environ['S3_ACCESS_KEY'],
aws_secret_access_key=os.environ['S3_SECRET_KEY'],
)
s3.upload_file('local-file.jpg', 'myapp-uploads', 'uploads/photo.jpg')
Generating Pre-Signed URLs (Secure Temporary Access)
Rather than making a bucket public, generate time-limited URLs granting temporary access to a specific file — useful for private user uploads that shouldn't be permanently public:
const { GetObjectCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');
const url = await getSignedUrl(s3, new GetObjectCommand({
Bucket: 'myapp-uploads',
Key: 'uploads/private-document.pdf',
}), { expiresIn: 3600 });
Direct Browser-to-Storage Uploads (Bypassing Your Server)
For large files, generate a pre-signed upload URL and have the browser upload directly to object storage, avoiding routing large file transfers through your application server entirely:
const { PutObjectCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');
const uploadUrl = await getSignedUrl(s3, new PutObjectCommand({
Bucket: 'myapp-uploads',
Key: `uploads/${filename}`,
}), { expiresIn: 300 });
// Send uploadUrl to the browser, which PUTs the file directly to it
Serving Uploaded Files
Either serve directly from object storage (if the bucket/objects are public), or proxy through your application for access-controlled files, checking authorization before generating a pre-signed URL or streaming the file.
Migrating Existing Local Uploads to Object Storage
aws s3 sync /var/www/myapp/uploads/ s3://myapp-uploads/ --profile minio --endpoint-url https://storage.yourdomain.com
Common Errors
"SignatureDoesNotMatch" — usually a clock synchronization issue between your application server and the storage endpoint, or incorrect credentials; verify NTP is enabled (see How to Set the Correct Timezone and Enable NTP on a Linux VPS).
Uploads fail for large files — check your web server's client_max_body_size (Nginx) if routing through your application, or use direct browser-to-storage uploads to bypass this limit entirely.
Best Practices
- Never hardcode storage credentials in application code — use environment variables
- Use pre-signed URLs for private content rather than making buckets fully public
- Consider direct browser uploads for large files to reduce load on your application server
Related Articles
- How to Set Up Self-Hosted S3-Compatible Object Storage with MinIO
- How to Manage Environment Variables and Secrets on a VPS
- How to Back Up to Object Storage (S3-Compatible)
