Release & Deployment
Estimated time: 15–30 min

Production Deployment Checklist

A stack-agnostic, step-by-step checklist to prepare, execute, verify, and safely complete production software deployments.

#Deployment#Production#Release#DevOps#Rollback#Monitoring#Data Safety
Change Risk Level:
Phase Preset:
Progress:0%0 of 38 completed

1. Pre-Deployment & Pre-Flight Readiness

Ensure code approvals, build artifacts, configuration secrets, and team notifications are ready.

0/6 checks
Change approvals and pull request sign-off confirmedBlockingRelease Lead
CI/CD pipeline and automated test suite passingBlockingDevOps / CI
Release artifact version and tag unambiguously identifiedBlockingDevOps
Production environment variables and secrets provisionedBlockingDevOps
Feature flags and initial rollout percentage configuredImportantProduct / Dev
On-call deployment lead and rollback owner designatedBlockingRelease Lead

2. Database & Data Safety

Safeguard schema migrations, lock avoidance, and data backfills.

0/6 checks
Backward-compatible schema migration designBlockingBackend Dev / DBA
Exclusive table lock duration assessedBlockingDBA
Non-blocking index creation specifiedImportantDBA
Data backfill scripts rate-limited & chunkedImportantBackend Dev
Database connection pool capacity verifiedImportantDevOps / DBA
Point-in-time recovery (PITR) & recent backup activeBlockingDBA / Ops

3. Rollback Readiness & Contingency

Establish objective revert criteria and test rollback mechanisms.

0/5 checks
Quantitative rollback trigger thresholds agreed uponBlockingRelease Lead
Previous stable version image tag ready for instant deployBlockingDevOps
Down-migration script verified or migration decoupledBlockingBackend Dev / DBA
Feature flag kill-switch verifiedImportantDev
Single designated decision lead for rollback callBlockingRelease Lead

4. Deployment Execution

Execute release strategy (rolling, green/blue, canary) cleanly.

0/5 checks
Target production environment verifiedBlockingDevOps
Pre-deployment database migrations executedBlockingDBA / DevOps
Rolling pod deployment / traffic shift initiatedBlockingDevOps
Container boot health & readiness probes passingBlockingDevOps
Zero pod crash-loops or exit status errorsBlockingDevOps

5. Live Verification & Telemetry Inspection

Verify actual production traffic performance and business functionality.

0/6 checks
Primary production domain and HTTP probes 200 OKBlockingQA / Dev
Critical user journey synthetic smoke test passedBlockingQA / Product
APM log streams inspected for new stack tracesBlockingDev / SRE
System metric baselines stable (CPU, Memory, Disk, DB)ImportantSRE / DevOps
Asynchronous background workers & job queues processingImportantBackend Dev
Third-party API integrations and webhooks operatingImportantBackend Dev

6. Phased Rollout & Traffic Shifting (Optional)

Ramp up user exposure safely for high-risk or canary releases.

0/5 checks
Initial canary traffic slice active (5% – 10%)ImportantDevOps / SRE
Canary telemetry soak period completed (15–30 mins)ImportantSRE
Progressive traffic ramp-up executed (25% -> 50% -> 100%)ImportantDevOps
Feature flag targeted rollout expandedSuggestionProduct Lead
Old target group / legacy pod instances decommissionedNitDevOps

7. Post-Deployment Cleanup & Sign-Off

Complete post-release soak, notify stakeholders, and log release records.

0/5 checks
30-minute telemetry soak period completedBlockingSRE / On-Call
Business conversion metrics and funnel stableImportantProduct / Data
Deployment channel and stakeholders notified of successImportantRelease Lead
Deployment record and release notes publishedSuggestionRelease Lead
Temporary rollout feature flag cleanup scheduledNitDev
Operational Definition

What "Deployment Complete" Actually Means

A deployment is not complete simply because the CI/CD pipeline finished or container pods report green. A release is only complete when live production verification proves system health under real user traffic.

1. Live Verification

Primary application domain URLs load cleanly, API endpoints respond with HTTP 200, and key user flows (login, payment, CRUD) pass synthetic checks.

2. Telemetry Stability

APM error rates remain below baseline thresholds (< 0.05%), p95/p99 latency is normal, log aggregators show no new stack trace spikes, and DB pools are healthy.

3. Post-Soak Sign-Off

The 30-minute post-release soak period finishes without memory leaks or queue backlogs, stakeholders are notified, and release records are updated.

Risk-Proportional Operations

Sizing Verification Depth to Deployment Risk

Match your pre-flight rigor and soak duration to the change risk. Select your risk level at the top of the checklist to filter relevant verification steps.

Low Risk5-Min Verification

Isolated Patches & UI

Small CSS tweaks, copy updates, bug fixes isolated to a single component without database or authentication involvement.

  • Quick smoke test against production URL
  • Basic HTTP status probe
Medium Risk15-Min Telemetry Soak

App Logic & Config Flags

New backend API routes, business logic modifications, environment variable updates, or third-party SDK version bumps.

  • Pre-flight secret store verification
  • APM log error inspection & latency check
High Risk30–60 Min Mandatory Gate

Schema, Auth & Infrastructure

Database migrations, authentication/session logic, payment gateways, core router changes, or infrastructure migrations.

  • Mandatory rollback gate & down-migration test
  • Table lock duration & DB snapshot check
Incident Prevention

Top 4 Production Deployment Failure Modes

Postmortems show that over 80% of production outages during deployments stem from four predictable errors:

1. Missing Environment Variables

Code is deployed that references a new environment variable before it is written to the production secret manager, causing instant container crash-loops on boot.

2. Exclusive Table Locks & Incompatible Schemas

Database migrations run `ALTER TABLE` with `NOT NULL` without default values, locking large production tables for minutes or breaking older running server instances.

3. Deploying Without a Reversible Rollback Plan

When an outage strikes, teams discover the previous release image tag is unknown or non-reversible database changes prevent reverting the application binary.

4. Unnoticed Asynchronous Queue & Worker Failures

Web HTTP endpoints return 200 OK while background event queues and worker threads crash silently, accumulating massive un-processed job backlogs.

Why the Post-Deployment Monitoring Period Matters

Many critical defects—such as slow memory leaks, database connection pool exhaustion, and edge-case exceptions—only manifest under high production user traffic over time.

30-Minute Telemetry Window

Maintain active observation of Datadog/Sentry/Prometheus dashboards for at least 30 minutes post-release before disbanding the deployment channel.

Compare Against Historical Baselines

Compare error rates, latency p99, and CPU/memory curves directly against pre-deployment baselines to catch subtle regressions.

When to Use This Checklist

  • Before initiating production release pipelines for web services, APIs, worker processes, or frontend bundles.
  • During scheduled maintenance windows, continuous deployment rollouts, or emergency hotfixes.
  • When deploying database schema changes, infrastructure modifications, or major dependency upgrades.
  • To guide team deployment handoffs and confirm live post-release verification.

Common Pitfalls to Avoid

  • Deploying without verifying environment variables or secrets exist in the production secret store.
  • Triggering database schema migrations that lock heavy tables or break compatibility with running app instances.
  • Lacking a clear, tested rollback plan or missing one-click revert access to the previous stable release artifact.
  • Declaring 'deployment complete' as soon as CI passes without verifying actual live user journeys and APM log streams.