@kennethsolomon/shipkit 1.0.0

package/skills/sk:schema-migrate/orms/sqlalchemy.md

@@ -0,0 +1,385 @@
+# /schema-migrate — SQLAlchemy + Alembic Analysis (Phases 2–5)
+
+> This file is loaded by `SKILL.md` when SQLAlchemy + Alembic is detected.
+> Execute Phase 2 through Phase 5 below, then return the final report.
+
+---
+
+## Phase 2: Classify Changes
+
+### Files to Scan (read in parallel)
+
+1. **`alembic.ini`** — Alembic configuration
+   - Extract `sqlalchemy.url` → determine dialect
+   - Extract `script_location` (default: `alembic/`)
+
+2. **`alembic/env.py`** — environment configuration
+   - Identify `target_metadata` import path → find model files
+   - Check `render_as_batch` setting (required for SQLite)
+   - Check if `include_schemas` or `compare_type` options are set
+
+3. **`alembic/versions/`** — migration history
+   - List all revision files sorted by `down_revision` chain
+   - Read the most recent revision file(s)
+   - Check for multiple heads (`# head revision` comments or `alembic heads` output)
+
+4. **Model files** (path from `env.py` import → `target_metadata`)
+   - Parse `Column()` definitions, types, constraints, indexes, FKs
+   - Track: table names, column types, nullable flags, defaults
+
+5. **`.env`** — environment config
+   - Fallback for `DATABASE_URL` if `alembic.ini` uses environment variable interpolation
+
+6. **`git diff HEAD -- [models_path]`** — uncommitted model changes
+
+7. **`git log --oneline -5 -- [models_path]`** — recent model commit history
+
+### Dialect Detection
+
+From `sqlalchemy.url` in `alembic.ini`:
+
+| URL Prefix | Dialect |
+|------------|---------|
+| `postgresql://` or `postgresql+psycopg2://` | PostgreSQL |
+| `mysql://` or `mysql+pymysql://` | MySQL |
+| `sqlite:///` | SQLite |
+| `mssql+pyodbc://` | SQL Server |
+
+### Risk Matrix
+
+| Change | Risk | Notes |
+|--------|------|-------|
+| Add new table (`op.create_table`) | 🟢 Safe | Additive — no existing data affected |
+| Add nullable column | 🟢 Safe | `nullable=True` or `server_default` present |
+| Add index (`op.create_index`) | 🟢 Safe | Non-blocking for most engines |
+| Add column with `server_default` | 🟡 Careful | Existing rows get default — verify acceptability |
+| Add unique constraint | 🟡 Careful | Fails if duplicate values exist in current data |
+| Add FK (`op.create_foreign_key`) | 🟡 Careful | Orphan row check required; SQLite: limited support |
+| Add NOT NULL column without `server_default` | 🔴 Breaking | Fails on any non-empty table |
+| Change column type (`op.alter_column`) | 🔴 Breaking | Use `postgresql_using` for casts; SQLite: table recreation |
+| Drop column (`op.drop_column`) | 🔴 Breaking | SQLite < 3.35: not supported natively |
+| Drop table (`op.drop_table`) | 🔴 Breaking | Destructive — check all references |
+| Rename column | 🔴 Breaking | Autogenerate blindspot: detected as drop+add, not rename |
+| Change nullable (`nullable=False`) | 🟡 Careful | Pre-check for NULL values before applying |
+
+### Also Detect
+
+- ⚠️ **Broken revision chain**: A revision file has a `down_revision` that doesn't exist → migration chain broken
+- ⚠️ **Multiple heads**: Two or more revisions with no `down_revision` pointing to them → `alembic heads` will show multiple
+- ⚠️ **`render_as_batch` missing**: SQLite env.py without `render_as_batch=True` → ALTER operations will fail
+- ⚠️ **Autogenerate blindspots**: Alembic autogenerate does NOT detect:
+  - Column server defaults (only `default=` in Python, not DB-level)
+  - Check constraints (`CheckConstraint`)
+  - Collation changes
+  - Sequence changes
+  - Column renames (shows as drop+add)
+
+---
+
+## Phase 3: Present Risk Report
+
+### Report Format
+
+```
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  /schema-migrate — Schema Change Analysis
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+ORM:              SQLAlchemy + Alembic
+Dialect:          PostgreSQL
+Workflow:         alembic upgrade head (versioned migrations)
+Script location:  alembic/
+Config:           alembic.ini
+Current head:     a3f2b1c9d8e4
+
+Status:
+  📁 Revision chain: INTACT (no broken links)
+  🔀 Heads: 1 (no merge needed)
+  📝 Models: MODIFIED (uncommitted changes)
+  🔧 render_as_batch: N/A (PostgreSQL)
+
+Detected Changes (since last commit):
+──────────────────────────────────────────
+
+🟢 SAFE (1 change)
+  • jobs: Added nullable column `applied_notes` (Text, nullable=True)
+
+🟡 CAREFUL (1 change — review before migrating)
+  • profile: Added column `email` (String(255), NOT NULL, server_default='')
+    ↳ All existing rows will get empty string value
+    ↳ Is '' an acceptable default? Consider nullable first.
+
+🔴 BREAKING (1 change — stop and read migration plan)
+  • jobs: Column `job_type` changed String → Integer
+    ↳ PostgreSQL: requires USING cast in ALTER COLUMN
+    ↳ Review autogenerated migration — add postgresql_using if missing
+    ↳ SQLite: full table recreation via batch operations
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```
+
+### Dialect-Specific Notes
+
+#### PostgreSQL
+- ✅ Full ALTER TABLE support — most changes are online
+- ⚠️ Column type changes require explicit `USING` clause for non-trivial casts
+- ⚠️ `op.alter_column` without `postgresql_using` will fail if data is incompatible
+- ✅ Migrations are versioned and fully reversible
+
+#### MySQL
+- ⚠️ Online DDL available in MySQL 8.0+
+- ⚠️ Some ALTER operations trigger full table reconstruction
+- ⚠️ Implicit type casts may fail silently
+
+#### SQLite
+- 🚨 `render_as_batch=True` MUST be set in `env.py` for any column modifications
+- 🚨 `DROP COLUMN` not supported in SQLite < 3.35 (Python 3.11 / SQLite 3.35+)
+- ✅ Batch operations recreate the table safely (data preserved)
+- ⚠️ FK support in SQLite is off by default — check `PRAGMA foreign_keys = ON`
+
+---
+
+## Phase 4: Per-Change Migration Plans
+
+### Scenario: Multiple Heads
+
+```
+Detection: alembic heads returns more than one revision
+
+Problem: Branched revision history — alembic upgrade head is ambiguous.
+
+Fix: Merge heads with a merge revision
+
+  # See current heads
+  alembic heads
+
+  # Merge two heads (replace rev1 and rev2 with actual IDs)
+  alembic merge -m "merge heads" rev1 rev2
+
+  # This creates a new migration file with both heads as down_revision:
+  # down_revision = ('rev1', 'rev2')
+
+  # Apply the merge
+  alembic upgrade head
+```
+
+---
+
+### Scenario: Add NOT NULL Column Without `server_default`
+
+```
+Problem: Non-empty table will fail — existing rows have no value for new column.
+
+Three-step safe migration:
+
+  Step 1: Add column as nullable
+    op.add_column('users', sa.Column('email', sa.String(255), nullable=True))
+
+  Step 2: Backfill existing rows (in same or separate migration)
+    op.execute("UPDATE users SET email = '' WHERE email IS NULL")
+
+  Step 3: Alter column to NOT NULL
+    op.alter_column('users', 'email', nullable=False)
+
+All three steps can be in a single revision file:
+
+  def upgrade():
+      op.add_column('users', sa.Column('email', sa.String(255), nullable=True))
+      op.execute("UPDATE users SET email = '' WHERE email IS NULL")
+      op.alter_column('users', 'email', nullable=False)
+
+  def downgrade():
+      op.alter_column('users', 'email', nullable=True)
+      op.drop_column('users', 'email')
+```
+
+---
+
+### Scenario: SQLite — `render_as_batch` Missing
+
+```
+Problem: Alembic cannot ALTER columns in SQLite without batch operations.
+
+Fix: Edit alembic/env.py to add render_as_batch=True
+
+  # In env.py, find the run_migrations_online() function:
+  # BEFORE:
+  context.configure(
+      connection=connection,
+      target_metadata=target_metadata,
+  )
+
+  # AFTER:
+  context.configure(
+      connection=connection,
+      target_metadata=target_metadata,
+      render_as_batch=True,  # ← add this line
+  )
+
+  # Also update run_migrations_offline() if present:
+  context.configure(
+      url=url,
+      target_metadata=target_metadata,
+      render_as_batch=True,  # ← add here too
+  )
+
+After this change, all ALTER operations will use SQLite's table recreation pattern:
+  with op.batch_alter_table('users') as batch_op:
+      batch_op.alter_column('email', nullable=False)
+```
+
+---
+
+### Scenario: Change Column Type on PostgreSQL
+
+```
+Problem: op.alter_column without USING clause fails if data is incompatible.
+
+Autogenerated migration (may be missing USING):
+  op.alter_column('jobs', 'job_type',
+      existing_type=sa.VARCHAR(),
+      type_=sa.INTEGER(),
+      existing_nullable=True)
+
+Fix: Add postgresql_using to the alter_column call:
+  op.alter_column('jobs', 'job_type',
+      existing_type=sa.VARCHAR(),
+      type_=sa.INTEGER(),
+      postgresql_using='job_type::integer',  # ← explicit CAST
+      existing_nullable=True)
+
+Pre-check for non-castable data:
+  SELECT job_type FROM jobs WHERE job_type !~ '^[0-9]+$';
+  # If rows found: these cannot cast to integer — fix data first
+
+For complex type changes (text → custom type, etc.):
+  # Use a temporary column approach:
+  # 1. Add new column with target type
+  # 2. Copy with explicit cast: UPDATE jobs SET job_type_new = CAST(job_type AS INT)
+  # 3. Drop old column
+  # 4. Rename new column
+```
+
+---
+
+### Scenario: Rename Column — Autogenerate Blindspot
+
+```
+Problem: Alembic autogenerate cannot detect renames — shows as drop+add.
+
+Detection from git diff:
+  - Model file: renamed attribute "old_name" → "new_name"
+  - Autogenerated migration: op.drop_column + op.add_column (DATA LOSS!)
+
+Fix: Write rename manually using op.alter_column:
+
+  def upgrade():
+      op.alter_column('users', 'old_name', new_column_name='new_name')
+
+  def downgrade():
+      op.alter_column('users', 'new_name', new_column_name='old_name')
+
+  # op.alter_column for rename is supported in PostgreSQL, MySQL, SQLite (batch)
+  # For SQLite, use batch operations:
+  with op.batch_alter_table('users') as batch_op:
+      batch_op.alter_column('old_name', new_column_name='new_name')
+
+Workflow:
+  1. alembic revision --autogenerate -m "rename_column"
+  2. Open generated file — replace drop+add with alter_column rename
+  3. alembic upgrade head
+```
+
+---
+
+## Phase 5: Command Recommendations
+
+### Core Commands
+
+```bash
+# Check current migration state
+alembic current
+
+# List all heads (should be 1)
+alembic heads
+
+# Show full migration history
+alembic history --verbose
+
+# Generate migration from model changes (autogenerate)
+alembic revision --autogenerate -m "[describe_change]"
+
+# Generate empty migration (for manual SQL)
+alembic revision -m "[describe_change]"
+
+# Apply all pending migrations
+alembic upgrade head
+
+# Apply one migration at a time
+alembic upgrade +1
+
+# Roll back one migration
+alembic downgrade -1
+
+# Roll back to specific revision
+alembic downgrade [revision_id]
+
+# Merge diverged heads
+alembic merge -m "merge heads" [rev1] [rev2]
+
+# Mark current DB state as head (without running migrations)
+alembic stamp head
+
+# Mark specific revision as applied
+alembic stamp [revision_id]
+```
+
+### Workflow: Development
+
+```bash
+# 1. Review pending model changes
+/schema-migrate
+
+# 2. Generate migration
+alembic revision --autogenerate -m "[describe_change]"
+
+# 3. Review generated migration file (ALWAYS review before applying!)
+cat alembic/versions/[latest_revision].py
+
+# 4. Edit if needed (add USING clauses, fix renames, etc.)
+
+# 5. Apply
+alembic upgrade head
+
+# 6. Verify state
+alembic current
+
+# 7. Commit
+git add alembic/versions/
+git commit -m "feat(schema): [describe change]"
+```
+
+### Workflow: Production Deployment
+
+```bash
+# 1. Check current state
+alembic current
+
+# 2. Preview pending migrations
+alembic history --verbose
+
+# 3. Apply
+alembic upgrade head
+
+# 4. Verify
+alembic current
+```
+
+### SQLite — Enabling Batch Operations
+
+```bash
+# After adding render_as_batch=True to env.py:
+alembic revision --autogenerate -m "column_change"
+# Verify generated migration uses batch_alter_table context manager
+alembic upgrade head
+```

package/skills/sk:schema-migrate/references/detection.md

@@ -0,0 +1,110 @@
+# /schema-migrate — ORM Detection Heuristics
+
+Used by `SKILL.md` Phase 1 to detect the active ORM and route to the correct `orms/[orm].md` file.
+
+---
+
+## Detection Priority Order
+
+First match wins. Check signals in this order:
+
+| Priority | ORM | Definitive Signal | Secondary Check |
+|----------|-----|-------------------|-----------------|
+| 1 | Drizzle | `drizzle.config.ts` or `drizzle.config.js` exists | `package.json` → `drizzle-orm` dep |
+| 2 | Prisma | `prisma/schema.prisma` exists | `package.json` → `@prisma/client` |
+| 3 | Laravel | `composer.json` → `laravel/framework` | `database/migrations/` exists |
+| 4 | SQLAlchemy | `alembic.ini` exists | `alembic/versions/` exists |
+| 5 | Rails | `Gemfile` → `rails` or `activerecord` gem | `db/migrate/` + `db/schema.rb` |
+
+---
+
+## Supabase Overlay
+
+Supabase is detected **alongside** the ORM, not instead of it.
+
+**Signal:** `supabase/config.toml` exists → set `isSupabase = true`
+
+Effect: The detected ORM file's Phase 5 commands will include Supabase CLI workflow in addition to the standard ORM commands. Supabase overlay applies primarily to Drizzle and Prisma projects (both use PostgreSQL under Supabase).
+
+---
+
+## Ambiguity Handling
+
+If two definitive signals match (e.g., monorepo with both `drizzle.config.ts` and `composer.json`), ask the user:
+
+> "Found both `drizzle.config.ts` (Drizzle ORM) and `composer.json` (Laravel). Which ORM should I analyze?"
+
+Do not guess — wait for explicit confirmation.
+
+---
+
+## Detection Flow (Phase 1 Steps)
+
+```
+Step 1: Read in parallel —
+        drizzle.config.ts, drizzle.config.js,
+        prisma/schema.prisma,
+        composer.json,
+        alembic.ini,
+        Gemfile,
+        package.json,
+        supabase/config.toml
+
+Step 2: Apply priority rules →
+        - Does drizzle.config.ts/js exist? → ORM = Drizzle
+        - Else does prisma/schema.prisma exist? → ORM = Prisma
+        - Else does composer.json have laravel/framework? → ORM = Laravel
+        - Else does alembic.ini exist? → ORM = SQLAlchemy
+        - Else does Gemfile have rails or activerecord? → ORM = Rails
+        - Else: Unknown — report error and ask user to specify
+
+        Supabase check (independent):
+        - Does supabase/config.toml exist? → isSupabase = true
+
+Step 3: Report detection result:
+        "Detected: [ORM] ([dialect]) — loading orms/[orm].md"
+        e.g., "Detected: Drizzle ORM (SQLite) — loading orms/drizzle.md"
+        e.g., "Detected: Prisma (PostgreSQL + Supabase) — loading orms/prisma.md"
+
+Step 4: Load orms/[orm].md and execute its Phase 2 through Phase 5
+```
+
+---
+
+## Dialect Detection (per ORM)
+
+After the ORM is identified, detect the database dialect:
+
+### Drizzle
+- Read `drizzle.config.ts` → `dialect` field: `sqlite` | `postgres` | `mysql`
+- Supabase: `dialect === 'postgres'` + URL contains `supabase`
+
+### Prisma
+- Read `prisma/schema.prisma` → `datasource db { provider = "..." }`
+- Values: `postgresql` | `mysql` | `sqlite` | `sqlserver` | `mongodb`
+- Note: MongoDB does not support migrations — flag this to the user
+
+### Laravel
+- Read `.env` → `DB_CONNECTION` value: `mysql` | `pgsql` | `sqlite`
+- Fallback: read `config/database.php` → `default` key
+- Default if undetectable: `mysql`
+
+### SQLAlchemy
+- Read `alembic.ini` → `sqlalchemy.url` value
+- Prefix: `postgresql` | `mysql` | `sqlite` | `mssql`
+
+### Rails
+- Read `config/database.yml` → `adapter:` value under `development:`
+- Values: `postgresql` | `mysql2` | `sqlite3`
+
+---
+
+## ORM → File Map
+
+| Detected ORM | File to Load |
+|--------------|-------------|
+| Drizzle | `orms/drizzle.md` |
+| Prisma | `orms/prisma.md` |
+| Laravel | `orms/laravel.md` |
+| SQLAlchemy (Alembic) | `orms/sqlalchemy.md` |
+| Rails (ActiveRecord) | `orms/rails.md` |