How to Roll Back a Bad Deployment Quickly

Every deployment process needs a fast, reliable rollback plan — because eventually, a deployment will introduce a problem that needs immediate reversal. This guide covers rollback strategies for different deployment approaches.

Why Rollback Speed Matters

The faster you can revert to a known-good state, the less impact a bad deployment has on users — a rollback plan designed and tested in advance is far faster than improvising one during an active incident.

Rollback Strategy 1 — Git-Based Rollback (Simple Deployments)

cd /var/www/myapp
git log --oneline -5
git reset --hard PREVIOUS_COMMIT_HASH
npm install --production
pm2 reload myapp

Simple and effective, but requires re-running the full deployment process (dependency install, build steps) which takes time.

Rollback Strategy 2 — Docker Image Tag Rollback (Fast)

If deploying via tagged Docker images (see How to Build a Simple CI/CD Pipeline with GitHub Actions):

docker pull myregistry/myapp:PREVIOUS_TAG
docker compose up -d

Much faster than a Git-based rollback, since the previous version is already built and ready to run — no rebuild step required.

Rollback Strategy 3 — Blue-Green Switch (Fastest)

If using blue-green deployment (see How to Set Up Blue-Green Deployment on a VPS), rollback is simply switching the Nginx upstream back to the previous environment:

sed -i 's/3002/3001/' /etc/nginx/sites-available/myapp
nginx -t && systemctl reload nginx

Near-instant, since the previous version is already running and warmed up.

Rollback Strategy 4 — Database Rollback

Application rollback alone isn't sufficient if a migration also ran. Options:

  • Run a corresponding "down" migration if your framework supports reversible migrations
  • Restore from a pre-deployment backup, if the migration was destructive and irreversible — see How to Back Up and Restore MySQL/MariaDB Databases

This is why designing migrations to be backward-compatible (see the database migration note in How to Set Up Blue-Green Deployment on a VPS) matters — it avoids needing a database rollback at all in most cases.

Building a Rollback Runbook in Advance

Document the exact rollback steps for your specific deployment method before you need them, including:

  • The exact commands to run
  • How to verify the rollback succeeded
  • Who to notify

Automating Rollback

sudo nano /usr/local/bin/rollback.sh
#!/bin/bash
PREVIOUS_TAG=$(cat /var/www/myapp/.last-deployed-tag)
docker pull myregistry/myapp:$PREVIOUS_TAG
cd /var/www/myapp
docker compose up -d
echo "Rolled back to $PREVIOUS_TAG at $(date)" >> /var/log/rollback.log

Track the currently and previously deployed version automatically as part of your normal deployment script, so rollback always has a known-good target without manual lookup.

Testing Your Rollback Process

Periodically practice an actual rollback (in staging, or during a planned low-risk window) to confirm the process works as documented — an untested rollback plan can fail exactly when you need it most.

Deciding When to Roll Back vs Fix Forward

SituationRecommended Action
Critical functionality broken, fix unclearRoll back immediately
Minor issue, fix is quick and well-understoodFix forward may be faster than rollback + redeploy
Database migration already applied and hard to reverseOften faster to fix forward than attempt a complex rollback

Common Errors

Rollback script fails because the previous version reference is missing — ensure your deployment process always records what was previously deployed before overwriting it.

Best Practices

  • Design your deployment method with rollback speed in mind from the start
  • Always track the previous deployed version/tag automatically
  • Test rollback procedures periodically, not just when facing a real incident

Continue Reading

Browse more articles in DevOps & CI/CD.

  • deployment rollback, incident response, release management, rollback strategy
  • 0 用戶發現這個有用
這篇文章有幫助嗎?

相關文章

How to Set Up a Self-Hosted GitHub Actions Runner on a VPS

GitHub Actions' hosted runners work well for most projects, but a self-hosted runner on your own...

How to Deploy Automatically on Git Push (Webhook-Based Deployment)

Automating deployment whenever you push to a specific branch removes the manual "SSH in and pull"...

How to Set Up Blue-Green Deployment on a VPS

Blue-green deployment runs two identical production environments — only one live at a time...

How to Use Ansible for Server Configuration Management

Ansible automates server configuration through simple, human-readable YAML files — letting...

Infrastructure as Code Basics: Managing VPS Config with Terraform

Terraform lets you define infrastructure (VPS instances, networks, DNS records) as code, applied...