@launchsecure/launch-kit 0.0.26 → 0.0.27

package/scaffolds/migrate-safety/docs/migrations-runbook.md

@@ -0,0 +1,172 @@
+# Migrations runbook
+
+Operational reference for the three-layer migration safety system. Read this before running a migration in prod, and again when something has gone wrong.
+
+## The three layers (recap)
+
+| Layer | What | Where defined |
+|---|---|---|
+| 1 — Process | Expand-and-contract: never combine "add + backfill + drop" in one migration. Two PRs, two deploys. | `CLAUDE.md` |
+| 2 — Tooling | `scripts/migrate-with-backup.sh` — pg_dump before every `prisma migrate`, abort if dump fails. | `scripts/migrate-with-backup.sh` |
+| 3 — In-migration SQL | Pre-flight count + abort-on-orphan + sidecar backup table before any column drop. | `CLAUDE.md` |
+
+Recovery never depends on a single layer. PITR (if your provider has it) is a fourth, continuous layer underneath.
+
+## Routine: running a migration
+
+### Local (against `.env.testing` or `.env`)
+
+```bash
+# Always use these — they wrap pg_dump around prisma migrate.
+npm run db:migrate                  # = prisma migrate dev (creates a new migration)
+npm run db:migrate:deploy           # = prisma migrate deploy (applies pending migrations)
+npm run db:backup                   # dump only, no migration (manual snapshot)
+```
+
+Backups land in `.backups/pre-migrate-<mode>-<YYYYMMDD-HHMMSS>-<git-sha>.sql.gz`. The directory is gitignored. Prune old ones manually when convenient.
+
+If your local `pg_dump` is older than the server, the wrapper auto-falls-back to a Docker container with `postgres:16`. Override the image if your server is on a different major:
+
+```bash
+PG_DUMP_DOCKER_IMAGE=postgres:15 PG_DUMP_VIA_DOCKER=1 npm run db:backup
+```
+
+### Production (Vercel build)
+
+The Vercel `build` script still runs `prisma migrate deploy` directly — the *prod backup* is decoupled from the Vercel build and runs on GitHub Actions instead (see `.github/workflows/backup-on-migration.yml`). Whenever a commit on `master` or `implementation` touches `prisma/migrations/**` or `prisma/schema.prisma`, the workflow runs the same wrapper in `backup-only` mode against `PROD_DATABASE_URL` and uploads the dump as a GHA artifact.
+
+**One-time setup before the workflow can run**:
+
+1. Repo → Settings → Secrets and variables → Actions → "New repository secret"
+2. Name: `PROD_DATABASE_URL`. Value: full Postgres URL of prod (with credentials).
+3. Confirm the workflow file is on `master`. The first migration commit afterward will trigger it.
+
+## Recovery
+
+### Find the right backup
+
+**Local backups** — listed by timestamp + commit SHA:
+
+```bash
+ls -lt .backups/
+```
+
+**Prod backups (GHA artifacts)**:
+
+- Web UI: repo → **Actions** tab → "Backup prod DB before migration deploys" workflow → click the run for the relevant commit → scroll to "Artifacts" at the bottom → download the `.zip`.
+- CLI:
+  ```bash
+  gh run list --workflow=backup-on-migration.yml --limit 10
+  gh run download <run-id> -n prod-db-backup-<commit-sha>
+  ```
+
+GHA artifacts are retained for 90 days by default (configurable in the workflow file).
+
+### Restore commands
+
+The dump format is identical regardless of where it came from (local or GHA), so the restore commands work for both. Decompress with `gunzip -c` (portable; macOS `zcat` does NOT decompress `.gz`).
+
+**1. Inspect a dump without restoring:**
+
+```bash
+gunzip -c .backups/pre-migrate-deploy-<ts>-<sha>.sql.gz | head -50
+gunzip -c .backups/pre-migrate-deploy-<ts>-<sha>.sql.gz | grep -E '^(CREATE TABLE|INSERT INTO "Project")' | head
+```
+
+**2. Restore to a fresh scratch database (for forensics, dry-run, or recovering a specific row):**
+
+```bash
+# Create a scratch DB (psql must point at a server you control; do NOT do this against prod)
+createdb scratch_restore
+# Or via psql: psql -c 'CREATE DATABASE scratch_restore' "$ADMIN_DB_URL"
+
+gunzip -c .backups/pre-migrate-deploy-<ts>-<sha>.sql.gz \
+  | psql "postgresql://USER:PASS@HOST:PORT/scratch_restore"
+```
+
+Now you have an exact copy of the pre-migration state. Inspect, copy specific rows back to prod, or run the migration against the scratch DB to verify what *would* happen.
+
+**3. Full prod rollback (last resort — see provider section below first):**
+
+```bash
+# DESTRUCTIVE — drops the existing schema and replaces it with the dumped contents.
+# Only do this if PITR is unavailable AND you've decided a full rollback is the right call.
+# The dump uses --no-owner --no-privileges, so Postgres roles are NOT modified.
+
+# 1. Take a current dump first (in case rollback itself goes wrong)
+DATABASE_URL="$PROD_DATABASE_URL" npm run db:backup
+
+# 2. Drop and recreate public schema
+psql "$PROD_DATABASE_URL" -c 'DROP SCHEMA public CASCADE; CREATE SCHEMA public;'
+
+# 3. Restore from the pre-migration dump
+gunzip -c .backups/pre-migrate-deploy-<ts>-<sha>.sql.gz | psql "$PROD_DATABASE_URL"
+
+# 4. Re-run prisma migrate so the migrations table reflects current state
+DATABASE_URL="$PROD_DATABASE_URL" npx prisma migrate resolve --applied <last-good-migration>
+```
+
+### Provider-specific recovery (preferred over manual restore when available)
+
+| Provider | Recovery feature | Notes |
+|---|---|---|
+| Neon | Branch from PITR | Console → Branches → "Create branch from a previous state" → pick a timestamp before the bad migration. Update `DATABASE_URL` to the new branch. Free tier: 7-day retention. |
+| Vercel Postgres | Same as Neon (it's Neon) | Use the Vercel dashboard or Neon console. |
+| Supabase | Daily snapshots (paid tiers) | Dashboard → Database → Backups. |
+| RDS / Aurora | Automated snapshots + PITR | AWS console. PITR up to backup retention period. |
+| Self-hosted | Whatever you set up | Hopefully `pg_basebackup` + WAL archiving. |
+
+**Rule of thumb**: provider PITR > GHA artifact > local `.backups/`. Try them in that order — PITR is closest to "the exact moment before things broke," GHA artifacts are the moment before deploy, local backups are whoever ran the last migration on their laptop.
+
+## Dry-running a migration before merging
+
+Before merging a destructive migration PR, restore a fresh GHA artifact (or a `pg_dump` you took manually against prod) into a scratch DB and run the migration there. Check row counts before/after. If the wrapper aborts on orphan check, the migration is broken — fix the SQL.
+
+```bash
+# Pull most recent prod backup
+gh run download $(gh run list --workflow=backup-on-migration.yml --limit 1 --json databaseId -q '.[0].databaseId') \
+  -n prod-db-backup-<sha> -D /tmp/prod-snapshot
+
+# Spin up a scratch DB and restore
+createdb migration_dryrun
+gunzip -c /tmp/prod-snapshot/*.sql.gz | psql "postgresql://localhost/migration_dryrun"
+
+# Apply the pending migration
+DATABASE_URL="postgresql://localhost/migration_dryrun" \
+  bash scripts/migrate-with-backup.sh deploy
+
+# Spot-check: did the rows you expected get backfilled?
+psql "postgresql://localhost/migration_dryrun" -c 'SELECT count(*) FROM "Project" WHERE ...;'
+```
+
+If the dry-run looks right, merge. If it doesn't, fix the migration and try again.
+
+## Security
+
+**Treat dump files like prod credentials.** They contain:
+
+- Hashed passwords (still attackable offline)
+- OAuth tokens stored in `Integration.metadata` — these are live credentials for Vercel, GitHub, Neon, etc. Anyone with the dump can impersonate the integration.
+- User PII (emails, names, anything stored in user-facing tables)
+
+**Therefore:**
+
+- `.backups/` is gitignored and must stay that way. Never commit a `.sql.gz` from this directory.
+- GHA artifacts inherit the repo's access — only collaborators on the repo can download them. If you make the repo public or add an external collaborator, they get historical artifacts too. Audit before changing access.
+- When sharing a dump for debugging, scrub the sensitive tables first (`Integration`, `Account`, `Session`, anything with tokens). A quick scrub:
+  ```bash
+  gunzip -c backup.sql.gz | grep -v '^COPY public."Integration"' | gzip > backup-scrubbed.sql.gz
+  ```
+  (This is a blunt instrument — it drops the entire `Integration` COPY block. Inspect before sharing.)
+- Rotate any tokens that may have been exposed if a dump leaks. Vercel/GitHub/Neon integration tokens can all be revoked from their respective dashboards.
+
+## Adding support for a new DB engine
+
+Today the wrapper supports Postgres only. To add MySQL/MongoDB/etc.:
+
+1. Add a case branch in `scripts/migrate-with-backup.sh` matching the URL scheme (`mysql://`, `mongodb://`, etc.).
+2. Use the appropriate dump tool (`mysqldump`, `mongodump`).
+3. Update the sanity check to recognize the new dump format.
+4. Update this runbook with restore commands for the new engine.
+
+The orchestration logic (dump → validate → migrate, abort on dump failure) stays identical.

package/scaffolds/migrate-safety/scripts/migrate-with-backup.sh

@@ -0,0 +1,294 @@
+#!/usr/bin/env bash
+#
+# scripts/migrate-with-backup.sh — single source of truth for "no migration without a backup"
+#
+# Modes:
+#   dev          → dump DB, then run `prisma migrate dev` (passes extra args through)
+#   deploy       → dump DB, then run `prisma migrate deploy`
+#   backup-only  → dump DB, do not migrate (used by GHA pre-deploy workflow)
+#
+# Env:
+#   DATABASE_URL          (required) — target DB. Wrapper does not load .env files;
+#                                      caller is responsible (matches Prisma's behavior).
+#   BACKUP_DIR            (default: .backups) — where the dump file is written.
+#   MIGRATE_SKIP_BACKUP   (default: 0) — set to 1 to skip the dump step entirely.
+#                                       Use sparingly — e.g. fresh shadow DB seeding
+#                                       where there is provably no data to lose.
+#   PG_DUMP_VIA_DOCKER    (default: auto) — 1=force docker, 0=force local, auto=try
+#                                          local first then fall back to docker on
+#                                          version mismatch or missing pg_dump.
+#   PG_DUMP_DOCKER_IMAGE  (default: auto-detect from server) — image used for docker
+#                                          fallback. When unset, the wrapper probes
+#                                          the server's major version (via local psql
+#                                          or `docker run postgres:latest psql`) and
+#                                          uses `postgres:<major>`. Set explicitly to
+#                                          override (e.g. internal hardened image).
+#   PG_PROBE_DOCKER_IMAGE (default: postgres:latest) — image used only for the
+#                                          version probe; psql is forward/backward
+#                                          compatible for SHOW server_version.
+#
+# Output:
+#   $BACKUP_DIR/pre-migrate-<mode>-<timestamp>-<git-sha>.sql.gz
+#   The dump uses --no-owner --no-privileges so it restores cleanly across providers.
+#
+# Retention:
+#   After a successful backup, the wrapper sweeps $BACKUP_DIR for files matching
+#   pre-migrate-*.sql.gz older than 30 days and deletes them. Best-effort —
+#   failures are non-fatal. Prod backups are GHA workflow artifacts with their
+#   own 90-day retention and are unaffected.
+#
+# Failure model:
+#   If the dump fails for any reason, the migration is NOT run, and any partial
+#   .sql.gz file is removed. Exit code is non-zero.
+#
+# Engine support:
+#   Today: PostgreSQL (URL schemes postgres:// or postgresql://).
+#   The dispatcher below makes adding mysql:// or other engines a one-case branch.
+
+set -euo pipefail
+
+# ─── Argument parsing ─────────────────────────────────────────────────────────
+
+mode="${1:-}"
+shift || true
+
+if [[ -z "$mode" ]]; then
+  echo "usage: $0 <dev|deploy|backup-only> [extra prisma args...]" >&2
+  exit 2
+fi
+
+case "$mode" in
+  dev|deploy|backup-only) ;;
+  *)
+    echo "error: mode must be 'dev', 'deploy', or 'backup-only' (got '$mode')" >&2
+    exit 2
+    ;;
+esac
+
+if [[ -z "${DATABASE_URL:-}" ]]; then
+  echo "error: DATABASE_URL is not set" >&2
+  exit 2
+fi
+
+backup_dir="${BACKUP_DIR:-.backups}"
+skip_backup="${MIGRATE_SKIP_BACKUP:-0}"
+
+# ─── Cleanup: remove partial dumps + temp files on any non-zero exit ──────────
+
+err_log="$(mktemp)"
+backup_file=""
+dump_validated=""
+
+_cleanup() {
+  local rc=$?
+  rm -f "$err_log"
+  if [[ $rc -ne 0 && -n "$backup_file" && -f "$backup_file" && -z "$dump_validated" ]]; then
+    rm -f "$backup_file"
+  fi
+  exit $rc
+}
+trap _cleanup EXIT
+
+# ─── 1. Take the backup (unless explicitly skipped) ───────────────────────────
+
+if [[ "$skip_backup" == "1" ]]; then
+  if [[ "$mode" == "backup-only" ]]; then
+    echo "error: MIGRATE_SKIP_BACKUP=1 with mode=backup-only is contradictory" >&2
+    exit 2
+  fi
+  echo "[migrate-with-backup] MIGRATE_SKIP_BACKUP=1 — skipping pre-flight dump"
+else
+  mkdir -p "$backup_dir"
+
+  ts="$(date +%Y%m%d-%H%M%S)"
+  sha="$(git rev-parse --short HEAD 2>/dev/null || echo nogit)"
+  backup_file="${backup_dir}/pre-migrate-${mode}-${ts}-${sha}.sql.gz"
+
+  # Dispatch by URL scheme so non-Postgres engines can be added later.
+  scheme="${DATABASE_URL%%:*}"
+  case "$scheme" in
+    postgres|postgresql)
+      # ── Strip Prisma-only query params that pg_dump (libpq) rejects ──
+      # Keep libpq params (sslmode, sslcert, application_name, etc.) so cloud DBs still connect.
+      sanitized_url="$DATABASE_URL"
+      query="${sanitized_url#*\?}"
+      if [[ "$query" != "$sanitized_url" ]]; then
+        base="${sanitized_url%%\?*}"
+        kept=""
+        IFS='&' read -ra _params <<< "$query"
+        for _p in "${_params[@]}"; do
+          _key="${_p%%=*}"
+          case "$_key" in
+            schema|pgbouncer|connection_limit|pool_timeout|socket_timeout|statement_cache_size)
+              ;; # Prisma-only — drop
+            *)
+              if [[ -n "$kept" ]]; then kept="${kept}&${_p}"; else kept="$_p"; fi
+              ;;
+          esac
+        done
+        if [[ -n "$kept" ]]; then
+          sanitized_url="${base}?${kept}"
+        else
+          sanitized_url="$base"
+        fi
+      fi
+
+      # Docker-mode URL: localhost/127.0.0.1 → host.docker.internal so a containerized
+      # client can reach a Postgres running on the host machine.
+      docker_url=$(printf '%s' "$sanitized_url" | sed -E 's#@(localhost|127\.0\.0\.1)([:/])#@host.docker.internal\2#')
+
+      mode_choice="${PG_DUMP_VIA_DOCKER:-auto}"
+      probe_image="${PG_PROBE_DOCKER_IMAGE:-postgres:latest}"
+
+      # ── Server-version detection (sets docker_image when not user-overridden) ──
+      _detect_server_major() {
+        # Returns Postgres major version (e.g. "17") on stdout, empty if probe fails.
+        local result=""
+        # Try local psql first (faster, no docker pull).
+        if command -v psql >/dev/null 2>&1; then
+          result=$(psql "$sanitized_url" -t -A -c "SHOW server_version" 2>/dev/null \
+                   | head -n 1 | sed -E 's/^[^0-9]*([0-9]+).*/\1/')
+        fi
+        # Fall back to a containerized psql when local isn't installed or can't connect.
+        if [[ -z "$result" ]] && command -v docker >/dev/null 2>&1 && docker info >/dev/null 2>&1; then
+          result=$(docker run --rm \
+                    --add-host=host.docker.internal:host-gateway \
+                    "$probe_image" \
+                    psql "$docker_url" -t -A -c "SHOW server_version" 2>/dev/null \
+                    | head -n 1 | sed -E 's/^[^0-9]*([0-9]+).*/\1/')
+        fi
+        printf '%s' "$result"
+      }
+
+      if [[ -n "${PG_DUMP_DOCKER_IMAGE:-}" ]]; then
+        docker_image="$PG_DUMP_DOCKER_IMAGE"
+        echo "[migrate-with-backup] using user-pinned docker image: ${docker_image}"
+      else
+        detected_major="$(_detect_server_major)"
+        if [[ -n "$detected_major" ]]; then
+          docker_image="postgres:${detected_major}"
+          echo "[migrate-with-backup] detected postgres ${detected_major}.x server → docker image: ${docker_image}"
+        else
+          # Probe failed — fall back to "latest" and let pg_dump's own error guide the user.
+          docker_image="postgres:latest"
+          echo "[migrate-with-backup] could not detect server version, using ${docker_image} (set PG_DUMP_DOCKER_IMAGE to override)" >&2
+        fi
+      fi
+
+      # ── Dump strategies ──
+
+      _try_local() {
+        if ! command -v pg_dump >/dev/null 2>&1; then return 127; fi
+        echo "[migrate-with-backup] dumping postgres (local pg_dump) → ${backup_file}"
+        pg_dump --no-owner --no-privileges "$sanitized_url" 2>"$err_log" | gzip > "$backup_file"
+      }
+
+      _try_docker() {
+        if ! command -v docker >/dev/null 2>&1; then return 127; fi
+        if ! docker info >/dev/null 2>&1; then return 126; fi
+        echo "[migrate-with-backup] dumping postgres (docker ${docker_image}) → ${backup_file}"
+        docker run --rm -i \
+          --add-host=host.docker.internal:host-gateway \
+          "$docker_image" \
+          pg_dump --no-owner --no-privileges "$docker_url" 2>"$err_log" \
+          | gzip > "$backup_file"
+      }
+
+      _hint_install() {
+        echo "" >&2
+        echo "Fix one of:" >&2
+        echo "  1. Install matching pg_dump locally:" >&2
+        echo "       brew install postgresql@${detected_major:-17} && brew link --force postgresql@${detected_major:-17}   # macOS" >&2
+        echo "       sudo apt-get install postgresql-client-${detected_major:-17}                                          # Debian/Ubuntu" >&2
+        echo "  2. Force docker fallback (auto-detects server version):" >&2
+        echo "       PG_DUMP_VIA_DOCKER=1 npm run db:backup" >&2
+        echo "  3. Pin a specific image (overrides auto-detection):" >&2
+        echo "       PG_DUMP_DOCKER_IMAGE=postgres:<major> npm run db:backup" >&2
+      }
+
+      case "$mode_choice" in
+        1)
+          if ! _try_docker; then
+            echo "error: PG_DUMP_VIA_DOCKER=1 but docker pg_dump failed" >&2
+            cat "$err_log" >&2
+            _hint_install
+            exit 1
+          fi
+          ;;
+        0)
+          if ! _try_local; then
+            echo "error: local pg_dump failed (PG_DUMP_VIA_DOCKER=0 disables docker fallback)" >&2
+            cat "$err_log" >&2
+            _hint_install
+            exit 1
+          fi
+          ;;
+        auto)
+          if _try_local; then
+            : # success
+          elif grep -qE "server version|version mismatch|command not found" "$err_log" 2>/dev/null \
+               || [[ ! -s "$backup_file" ]]; then
+            local_err="$(cat "$err_log")"
+            echo "[migrate-with-backup] local pg_dump unusable; trying docker fallback..." >&2
+            [[ -n "$local_err" ]] && echo "  (local error: ${local_err})" >&2
+            : > "$backup_file" # reset partial output before retry
+            if ! _try_docker; then
+              echo "error: docker fallback failed" >&2
+              cat "$err_log" >&2
+              _hint_install
+              exit 1
+            fi
+          else
+            echo "error: pg_dump failed:" >&2
+            cat "$err_log" >&2
+            exit 1
+          fi
+          ;;
+        *)
+          echo "error: PG_DUMP_VIA_DOCKER must be 0, 1, or auto (got '$mode_choice')" >&2
+          exit 2
+          ;;
+      esac
+
+      # ── Sanity check ──
+      # gzip must produce a non-empty file with a real pg_dump header. Capture the
+      # first chunk into a variable rather than chaining `gunzip | head | grep`:
+      # under `pipefail`, `head` closing early sends SIGPIPE to gunzip, making the
+      # pipeline return non-zero even when grep matched. Capturing breaks the pipe.
+      # Use `gunzip -c` (portable) — macOS `zcat` is BSD compress, doesn't decompress gzip.
+      header_block="$(gunzip -c "$backup_file" 2>/dev/null | head -10 || true)"
+      if [[ ! -s "$backup_file" ]] || ! grep -q "PostgreSQL database dump" <<<"$header_block"; then
+        echo "error: dump file is empty or missing PostgreSQL header — refusing to migrate" >&2
+        echo "       backup_file: $backup_file ($(du -h "$backup_file" 2>/dev/null | cut -f1 || echo 'missing'))" >&2
+        exit 1
+      fi
+      dump_validated=1
+      ;;
+    *)
+      echo "error: unsupported DATABASE_URL scheme '${scheme}://' — wrapper currently only supports postgres" >&2
+      echo "       to add support, extend the case statement in $(basename "$0")" >&2
+      exit 2
+      ;;
+  esac
+
+  size="$(du -h "$backup_file" | cut -f1)"
+  echo "[migrate-with-backup] dump complete (${size}) → ${backup_file}"
+
+  # ─── Prune local backups older than 30 days ────────────────────────────────
+  # Best-effort, non-fatal. Prod backups live as GHA artifacts with their own
+  # 90-day retention; this only sweeps the local .backups/ dir.
+  pruned=$(find "$backup_dir" -maxdepth 1 -type f -name 'pre-migrate-*.sql.gz' -mtime +30 -print -delete 2>/dev/null | wc -l | tr -d ' ' || true)
+  if [[ "${pruned:-0}" -gt 0 ]]; then
+    echo "[migrate-with-backup] pruned ${pruned} backup(s) older than 30 days"
+  fi
+fi
+
+# ─── 2. Run the migration (skipped in backup-only mode) ───────────────────────
+
+if [[ "$mode" == "backup-only" ]]; then
+  echo "[migrate-with-backup] backup-only mode — skipping prisma migrate"
+  exit 0
+fi
+
+echo "[migrate-with-backup] running: npx prisma migrate ${mode} $*"
+exec npx prisma migrate "$mode" "$@"