@evertonferreira96/agents 0.0.0-stage → 0.1.0

package/.claude/skills/postgres-best-practices/references/major-version-upgrades.md

@@ -0,0 +1,334 @@
+# Major Version Upgrades Reference
+
+## Contents
+- Upgrade methods overview
+- pg_upgrade (in-place)
+- Logical replication (minimal downtime)
+- Pre-upgrade checklist
+- Post-upgrade checklist
+- Testing strategy
+
+## Upgrade Methods Overview
+
+| Method | Downtime | Complexity | Rollback | Best for |
+|--------|----------|------------|----------|----------|
+| `pg_upgrade` | Minutes to hours | Low | Restore from backup | Most upgrades, moderate database sizes |
+| `pg_upgrade --swap` (PG18+) | Minutes | Low | Restore from backup | Fastest in-place upgrade |
+| Logical replication | Seconds | High | Switch back to old primary | Large databases requiring near-zero downtime |
+| pg_dump/pg_restore | Hours to days | Low | Old cluster still running | Small databases, or when other methods fail |
+
+**Default recommendation**: `pg_upgrade` for most cases. Use logical replication only when you need near-zero downtime on a large database.
+
+## pg_upgrade (In-Place)
+
+`pg_upgrade` replaces the old cluster's data files with the new version in place, without dumping and reloading data. PG18 supports five transfer modes:
+
+| Mode | Flag | Speed | Disk usage |
+|------|------|-------|------------|
+| Copy | (default) | Moderate | 2x disk during upgrade |
+| Copy file range | `--copy-file-range` (PG18+) | Fast where supported | 2x disk during upgrade |
+| Link | `--link` | Fast | Minimal extra disk (hard links) |
+| Swap | `--swap` (PG18+) | Fastest | No extra disk (swaps data dirs) |
+| Clone | `--clone` | Fast (if filesystem supports reflinks) | Minimal extra disk |
+
+### Step-by-Step: pg_upgrade
+
+#### 1. Install the New Version
+
+Install the new PostgreSQL version alongside the old one. Do not remove the old version yet.
+
+```bash
+# Example: upgrading from PG16 to PG17
+# Install PG17 (method depends on your OS/package manager)
+# Both versions coexist with different binary directories
+```
+
+#### 2. Run Pre-Upgrade Check
+
+```bash
+# Dry run — checks compatibility without making changes
+pg_upgrade \
+    --old-datadir /var/lib/postgresql/16/main \
+    --new-datadir /var/lib/postgresql/17/main \
+    --old-bindir /usr/lib/postgresql/16/bin \
+    --new-bindir /usr/lib/postgresql/17/bin \
+    --check
+```
+
+Fix any issues reported before proceeding. Common issues:
+- Extensions not available in the new version
+- Custom data types with incompatible binary formats
+- `contrib` modules that need updating
+
+#### 3. Stop the Old Cluster
+
+```bash
+pg_ctl stop -D /var/lib/postgresql/16/main
+```
+
+#### 4. Run pg_upgrade
+
+```bash
+# Copy mode (safe, uses more disk)
+pg_upgrade \
+    --old-datadir /var/lib/postgresql/16/main \
+    --new-datadir /var/lib/postgresql/17/main \
+    --old-bindir /usr/lib/postgresql/16/bin \
+    --new-bindir /usr/lib/postgresql/17/bin
+
+# Link mode (faster, old cluster becomes unusable)
+pg_upgrade \
+    --old-datadir /var/lib/postgresql/16/main \
+    --new-datadir /var/lib/postgresql/17/main \
+    --old-bindir /usr/lib/postgresql/16/bin \
+    --new-bindir /usr/lib/postgresql/17/bin \
+    --link
+
+# PG18+: Swap mode (fastest, swaps data directories)
+pg_upgrade \
+    --old-datadir /var/lib/postgresql/17/main \
+    --new-datadir /var/lib/postgresql/18/main \
+    --old-bindir /usr/lib/postgresql/17/bin \
+    --new-bindir /usr/lib/postgresql/18/bin \
+    --swap
+
+# Parallel checking: speed up pre-checks
+pg_upgrade ... --jobs 4
+```
+
+#### 5. Start the New Cluster
+
+```bash
+pg_ctl start -D /var/lib/postgresql/17/main
+```
+
+#### 6. Post-Upgrade Tasks
+
+After a successful upgrade, refresh optimizer statistics and remove the old cluster only after verification:
+
+```bash
+# Update optimizer statistics in stages
+vacuumdb --all --analyze-in-stages
+
+# Generated by pg_upgrade — delete old cluster data (only after confirming the upgrade works)
+./delete_old_cluster.sh
+```
+
+Current `pg_upgrade` releases generate the deletion script but may instruct you to run `vacuumdb` directly instead of generating `analyze_new_cluster.sh`.
+
+### Optimizer Statistics Preservation (PG18+)
+
+PG18's `pg_upgrade` preserves most optimizer statistics by default, reducing the post-upgrade performance dip:
+
+```bash
+# Default in PG18: statistics are preserved
+pg_upgrade ...
+
+# Opt out (if you want fresh statistics)
+pg_upgrade ... --no-statistics
+```
+
+Some statistics, including cumulative statistics and statistics that cannot be transferred, still need to be rebuilt. On older versions, always run `ANALYZE` on all databases immediately after upgrading.
+
+## Logical Replication (Minimal Downtime)
+
+For near-zero downtime upgrades on large databases, use logical replication to stream changes from the old version to the new version, then cut over.
+
+### Overview
+
+1. Set up new cluster on the target PG version
+2. Create schema on the new cluster
+3. Set up logical replication (old → new)
+4. Wait for initial sync + streaming to catch up
+5. Cut over applications to the new cluster
+
+See `logical-replication.md` for the full live migration pattern.
+
+### Advantages Over pg_upgrade
+
+- Downtime measured in seconds (just the application switchover)
+- Can upgrade across multiple major versions in one step
+- Allows testing the new cluster while the old one is still serving traffic
+- Rollback is simple: point applications back to the old cluster
+
+### Disadvantages
+
+- More complex setup
+- Sequences must be synchronized manually
+- DDL is not replicated — schema must be created manually on the target
+- Large objects (lo) are not replicated
+- Some extensions may behave differently across versions
+
+### pg_createsubscriber (PG17+)
+
+Converts a physical standby into a logical subscriber, simplifying the setup:
+
+```bash
+# PG18+: --all flag converts all databases at once
+pg_createsubscriber \
+    --pgdata /var/lib/postgresql/18/main \
+    --publisher-server "host=old_primary dbname=mydb" \
+    --all
+```
+
+## Pre-Upgrade Checklist
+
+### 1. Review Release Notes
+
+Read the release notes for **every version** between your current and target version. Pay attention to:
+- Removed features or changed defaults
+- Extension compatibility changes
+- Authentication changes (e.g., PG18 deprecates md5)
+- Behavior changes that could affect queries
+
+### 2. Check Extension Compatibility
+
+```sql
+-- List installed extensions and versions
+SELECT extname, extversion FROM pg_extension ORDER BY extname;
+
+-- Check if extensions are available in the new version
+-- (run against the new cluster after initdb)
+SELECT name, default_version FROM pg_available_extensions WHERE name IN (
+    SELECT extname FROM pg_extension
+) ORDER BY name;
+```
+
+### 3. Check for Deprecated Features
+
+```sql
+-- Check for md5 passwords (deprecated in PG18)
+SELECT rolname
+FROM pg_authid
+WHERE rolcanlogin AND rolpassword LIKE 'md5%';
+
+-- Check for removed/renamed GUC settings
+-- (pg_upgrade --check will catch these)
+```
+
+### 4. Take a Full Backup
+
+**Non-negotiable**. Before any upgrade:
+
+```bash
+# Full cluster backup
+pg_dumpall -f pre_upgrade_backup.sql
+
+# Or physical backup
+pg_basebackup -D /backup/pre_upgrade -Ft -z -Xs -P
+```
+
+### 5. Test on a Clone
+
+Never upgrade production first. Test on a copy:
+
+```bash
+# Create a test copy
+pg_basebackup -D /tmp/upgrade_test -Fp -Xs -P
+
+# Initialize an empty target cluster with the new version's binaries
+/usr/lib/postgresql/17/bin/initdb -D /tmp/new_cluster
+
+# Run pg_upgrade --check against the copy
+/usr/lib/postgresql/17/bin/pg_upgrade --check \
+    --old-datadir /tmp/upgrade_test \
+    --new-datadir /tmp/new_cluster \
+    --old-bindir /usr/lib/postgresql/16/bin \
+    --new-bindir /usr/lib/postgresql/17/bin
+```
+
+### 6. Plan for Replication
+
+If using streaming replication:
+- Standbys must be rebuilt after pg_upgrade (they can't follow a pg_upgraded primary)
+- Alternative: upgrade standbys with `pg_upgrade` too (stop all, upgrade all, restart)
+- Or use `pg_createsubscriber` (PG17+) to convert to logical replication
+
+## Post-Upgrade Checklist
+
+### 1. Update Optimizer Statistics
+
+```bash
+# Rebuild optimizer statistics in stages
+vacuumdb --all --analyze-in-stages --jobs 4
+```
+
+On PG18+ with `pg_upgrade`, most optimizer statistics are preserved by default, but analyze tables whose statistics were not transferred and tables changed after the upgrade.
+
+### 2. Check for Invalid Indexes
+
+```sql
+SELECT indexrelid::regclass, indisvalid
+FROM pg_index
+WHERE NOT indisvalid;
+```
+
+### 3. Update Extensions
+
+```sql
+-- Update extensions to latest available version
+ALTER EXTENSION pg_stat_statements UPDATE;
+ALTER EXTENSION postgis UPDATE;
+-- Repeat for each extension
+```
+
+### 4. Review Changed Defaults
+
+Check if new default values for GUC parameters affect your workload:
+
+```sql
+-- Compare settings that differ from defaults
+SELECT name, setting, boot_val, source
+FROM pg_settings
+WHERE setting != boot_val
+    AND source != 'default'
+ORDER BY name;
+```
+
+### 5. Monitor Performance
+
+After upgrade, monitor for:
+- Query plan regressions (optimizer changes across versions)
+- Connection behavior changes
+- Extension behavior changes
+
+```sql
+-- Compare top queries before/after via pg_stat_statements
+SELECT left(query, 80), calls,
+    round(mean_exec_time::numeric, 1) AS avg_ms
+FROM pg_stat_statements
+WHERE dbid = (SELECT oid FROM pg_database WHERE datname = current_database())
+ORDER BY total_exec_time DESC
+LIMIT 20;
+```
+
+### 6. Clean Up
+
+```bash
+# Remove old cluster (only after confirming upgrade is stable)
+./delete_old_cluster.sh
+
+# Or manually:
+rm -rf /var/lib/postgresql/16/main  # be very careful with this
+
+# Uninstall old PostgreSQL version
+# (method depends on your package manager)
+```
+
+## Testing Strategy
+
+### Regression Testing
+
+1. **Dump queries from pg_stat_statements** before upgrade
+2. **Replay on the test cluster** (or use `pg_stat_statements` to compare plans)
+3. **Compare EXPLAIN plans** for your top-20 queries by total time
+4. **Run application test suite** against the new version
+5. **Load test** at production-like traffic levels
+
+### Rollback Plan
+
+- **pg_upgrade (copy mode)**: Old cluster is intact — start it back up
+- **pg_upgrade (link mode)**: Old cluster is unusable — restore from backup
+- **pg_upgrade (swap mode)**: The file transfer is destructive; restore the old cluster from backup
+- **Logical replication**: Point applications back to the old cluster
+- **Always have a tested backup** regardless of method