agileflow 4.0.0-alpha.2 → 4.0.0-alpha.21

package/content/plugins/migration/plugin.yaml

@@ -0,0 +1,17 @@
+id: migration
+name: Migration
+description: Migration skill pack.
+version: 1.0.0
+enabledByDefault: false
+
+depends: []
+
+provides:
+  skills:
+    - id: agileflow-migration
+      dir: skills/agileflow-migration
+  agents:
+    - id: datamigration
+      path: agents/datamigration.md
+  hooks: []
+  templates: []

package/content/plugins/migration/skills/agileflow-migration/SKILL.md

@@ -0,0 +1,106 @@
+---
+name: agileflow-migration
+version: 1.0.0
+category: agileflow/migration
+description: |
+  Use when the user is migrating between frameworks, languages, library
+  versions, or data schemas. Plans zero-downtime migrations, generates
+  codemods, validates output, and produces rollback strategies.
+triggers:
+  keywords:
+    - migrate
+    - migration
+    - upgrade
+    - upgrade version
+    - move from
+    - switch to
+    - refactor to
+    - data migration
+    - zero downtime
+    - codemod
+    - breaking changes
+    - upgrade path
+  priority: 55
+provides:
+  agents: []
+learns:
+  enabled: true
+  file: _learnings/migration.yaml
+  maxEntries: 20
+depends:
+  skills: []
+  plugins: [migration]
+---
+
+# AgileFlow Migration
+
+Migration planning and execution assistant. Handles framework upgrades,
+library version bumps, data schema migrations, and language transitions
+— with validation, rollback strategies, and automated codemods where
+possible.
+
+## When this skill activates
+
+- User wants to upgrade a framework or library version
+- User is moving between technologies (e.g. CJS → ESM, REST → GraphQL)
+- User needs to migrate data between schemas or databases
+- User asks about breaking changes or upgrade paths
+- User mentions "zero downtime" or "rollback strategy"
+
+## Migration workflow
+
+| Step        | Command                       | What it does                             |
+| ----------- | ----------------------------- | ---------------------------------------- |
+| 1. Scan     | `/agileflow:migrate:scan`     | Identify all affected files and patterns |
+| 2. Plan     | `/agileflow:migrate:plan`     | Generate step-by-step migration plan     |
+| 3. Codemods | `/agileflow:migrate:codemods` | Auto-transform code where possible       |
+| 4. Validate | `/agileflow:migrate:validate` | Verify migration output is correct       |
+| Full        | `/agileflow:migrate`          | Run all steps                            |
+
+## Data migrations
+
+For database schema migrations, use `agileflow-datamigration` which covers:
+
+- Zero-downtime strategies (expand/contract pattern)
+- Backfill with concurrent write safety
+- Validation queries before and after
+- Rollback SQL generation
+
+## Safety rules
+
+- Always scan before planning — scope surprises are common
+- For data migrations: test on a copy first, validate row counts
+- For framework upgrades: check for behavioral changes, not just syntax
+- Always generate a rollback plan before executing
+
+## Integration
+
+- **agileflow-research** — run before planning a migration to gather release notes, breaking change logs, and community-documented gotchas for the specific version jump
+- **agileflow-adr** — document the migration decision (why this version, why this strategy, what was rejected) before executing; ADR is the runbook preface
+- **agileflow-planning** — use for impact analysis to understand which files, tests, and APIs are affected before generating the migration plan
+- **agileflow-database** — delegate schema-level migration steps (column renames, index changes, foreign key adjustments) to database expert
+- **agileflow-test-writer** — generate pre/post migration tests and data validation scripts to confirm row counts and aggregate consistency
+- **agileflow-audit** — run after migration completes to catch regressions introduced by the version change (API contracts, security headers, deprecated patterns)
+- **agileflow-engineering** — delegate the actual codemods and file changes when the migration scope is large; migration plans, engineering executes
+- **agileflow-delivery** — coordinate migration execution with deployment; zero-downtime migrations require delivery-level orchestration (feature flags, blue/green)
+- **agileflow-docs** — update installation guides, API docs, and README after the migration changes public-facing interfaces or configuration
+
+## References
+
+Load these files when you need deeper context for the relevant task:
+
+| File                                         | When to load                                                                                      |
+| -------------------------------------------- | ------------------------------------------------------------------------------------------------- |
+| `references/migration-patterns.md`           | Choosing a migration strategy — big bang vs strangler fig vs parallel run vs blue/green           |
+| `references/data-validation-checklist.md`    | Validating a migration — pre/post row counts, financial aggregates, cutover criteria              |
+| `references/rollback-playbook.md`            | Planning or executing a rollback — decision tree, communication templates, post-mortem structure  |
+| `references/version-compatibility-matrix.md` | Upgrading a library or framework — semver rules, breaking change detection, deprecation timelines |
+
+## Workflows
+
+Follow these step-by-step when the user initiates the matching action:
+
+| File                    | When to follow                                                                            |
+| ----------------------- | ----------------------------------------------------------------------------------------- |
+| `workflows/plan.md`     | User wants a migration plan — scans affected files, generates step-by-step strategy       |
+| `workflows/validate.md` | User wants to validate a migration — runs pre/post checks, compares counts and aggregates |

package/content/plugins/migration/skills/agileflow-migration/references/data-validation-checklist.md

@@ -0,0 +1,154 @@
+# Data Validation Checklist
+
+**Load this when:** validating data after a migration, designing pre/post
+migration checks, or building a data quality gate before cutover.
+
+## Pre-migration baseline (capture before running)
+
+Run these queries and save the output. Compare against post-migration results.
+
+### Row counts
+
+```sql
+-- Total rows per table
+SELECT table_name, n_live_tup AS row_count
+FROM pg_stat_user_tables
+ORDER BY n_live_tup DESC;
+
+-- Or for key tables explicitly
+SELECT
+  (SELECT COUNT(*) FROM users) AS users,
+  (SELECT COUNT(*) FROM orders) AS orders,
+  (SELECT COUNT(*) FROM transactions) AS transactions;
+```
+
+### Aggregate values (financial / critical)
+
+```sql
+-- Sum of money — must not change
+SELECT
+  SUM(amount) AS total_amount,
+  SUM(refunded_amount) AS total_refunds,
+  COUNT(*) FILTER (WHERE status = 'completed') AS completed_orders
+FROM orders;
+```
+
+### Status distributions
+
+```sql
+-- Status counts — proportions should be preserved
+SELECT status, COUNT(*) FROM orders GROUP BY status ORDER BY status;
+SELECT verified, COUNT(*) FROM users GROUP BY verified;
+```
+
+### Relationship integrity
+
+```sql
+-- Orphaned records (should be 0)
+SELECT COUNT(*) FROM order_items oi
+  LEFT JOIN orders o ON oi.order_id = o.id
+  WHERE o.id IS NULL;
+```
+
+## Post-migration validation
+
+Run the same queries. Compare:
+
+| Check                          | Expected      | Action if wrong                          |
+| ------------------------------ | ------------- | ---------------------------------------- |
+| Row counts                     | Match exactly | Investigate lost/duplicate rows          |
+| Financial sums                 | Match exactly | STOP — data loss, rollback               |
+| Status distributions           | Match exactly | Check transformation logic               |
+| Orphaned records               | 0             | Find FK violation, fix before proceeding |
+| Null counts in required fields | 0             | Backfill missing values                  |
+
+## Column-level validation
+
+```sql
+-- Check for unexpected nulls in columns that should be populated
+SELECT
+  COUNT(*) FILTER (WHERE email IS NULL) AS null_email,
+  COUNT(*) FILTER (WHERE created_at IS NULL) AS null_created_at,
+  COUNT(*) FILTER (WHERE user_id IS NULL) AS null_user_id
+FROM orders;
+
+-- Check for values outside expected ranges
+SELECT COUNT(*) FROM transactions
+  WHERE amount < 0 OR amount > 1000000;
+
+-- Check for format violations
+SELECT COUNT(*) FROM users
+  WHERE email NOT LIKE '%@%.%';
+
+-- Check for duplicate unique values
+SELECT email, COUNT(*) FROM users
+  GROUP BY email HAVING COUNT(*) > 1;
+```
+
+## Sampling validation
+
+For large datasets where full comparison isn't practical:
+
+```sql
+-- Sample 100 random rows, verify they look correct
+SELECT * FROM users ORDER BY RANDOM() LIMIT 100;
+
+-- Sample from each status bucket
+SELECT * FROM (
+  SELECT *, ROW_NUMBER() OVER (PARTITION BY status ORDER BY RANDOM()) AS rn
+  FROM orders
+) t WHERE rn <= 10;
+```
+
+Manual review of samples catches transformation bugs that aggregate checks miss.
+
+## Application-level validation
+
+After schema migration, verify application behavior:
+
+```
+⬜ New writes succeed (create a test record, verify it persists)
+⬜ Reads return expected data (fetch a known record, compare fields)
+⬜ Indexes are being used (EXPLAIN ANALYZE on key queries)
+⬜ Existing API endpoints return 200 (not 500 from schema mismatch)
+⬜ Background jobs complete without errors
+⬜ Reports/aggregations produce correct output
+```
+
+## Cutover decision criteria
+
+Only proceed to final cutover when ALL of these are true:
+
+```
+⬜ Row counts match (tolerance: 0 for critical tables, < 0.01% for large tables)
+⬜ All financial aggregates match exactly
+⬜ Zero orphaned records
+⬜ Zero unexpected nulls in required columns
+⬜ Sample review passes (100+ records spot-checked)
+⬜ Application smoke test passes
+⬜ Rollback has been tested in staging
+⬜ Team is available to monitor for 2 hours post-cutover
+```
+
+## Common migration data bugs
+
+| Bug                            | Symptom                       | How to detect               |
+| ------------------------------ | ----------------------------- | --------------------------- |
+| Off-by-one in batch processing | Last N rows missing           | Row count check             |
+| Timezone conversion error      | Timestamps shifted by hours   | Sample date comparison      |
+| Character encoding issue       | Garbled names/text            | Spot check non-ASCII values |
+| Duplicate row insertion        | Inflated counts               | Aggregate sum comparison    |
+| Null coalesce wrong default    | All nulls → 0 instead of NULL | Null count check            |
+| FK constraint skipped          | Orphaned records              | Orphan count check          |
+| Truncated strings              | Data cut off at 255 chars     | Check max-length values     |
+
+## Post-cutover monitoring (first 24 hours)
+
+Watch these metrics for anomalies:
+
+- Error rates in application logs (spike = schema mismatch)
+- Query latency (missing index = slow queries)
+- Database CPU and lock waits (backfill still running?)
+- Failed job counts (workers hitting migration-related errors)
+
+Set alert thresholds before cutover, not after.

package/content/plugins/migration/skills/agileflow-migration/references/migration-patterns.md

@@ -0,0 +1,209 @@
+# Migration Patterns
+
+**Load this when:** planning a zero-downtime migration, choosing between
+migration strategies, or designing a rollback plan.
+
+## The four migration strategies
+
+| Strategy          | When to use                          | Risk   | Complexity |
+| ----------------- | ------------------------------------ | ------ | ---------- |
+| **Big bang**      | Small dataset, maintenance window OK | High   | Low        |
+| **Strangler fig** | Live traffic, large codebase         | Low    | High       |
+| **Parallel run**  | Data integrity critical              | Low    | High       |
+| **Blue/green**    | Full deployment swap                 | Medium | Medium     |
+
+### Big bang
+
+Stop everything. Migrate. Restart. Simple but requires downtime.
+
+```
+Old system → [Maintenance window] → New system
+```
+
+**Use when:** Internal tools, staging environments, datasets < 10K rows, sub-1-hour migration.
+
+**Never for:** User-facing production systems without a maintenance page + user communication.
+
+### Strangler fig
+
+Route traffic gradually from old to new. Old system "dies" as new handles more.
+
+```
+All traffic → Old system
+↓ (add routing layer)
+Traffic → Router → Old system (90%) + New system (10%)
+↓ (shift gradually)
+Traffic → Router → New system (100%)
+↓ (remove old)
+Traffic → New system
+```
+
+**Use when:** Migrating a monolith to microservices, rewriting a module that can't go down.
+
+**Strangler fig checklist:**
+
+```
+⬜ Routing layer in place before migrating any traffic
+⬜ Both systems can handle same requests (parity check)
+⬜ Observability on both paths (latency, error rate)
+⬜ Feature flags control traffic split
+⬜ Rollback = flip flag back, not a deployment
+```
+
+### Parallel run
+
+Both systems run simultaneously. Write to both, compare outputs, switch reads when confident.
+
+```
+Writes → Old + New (dual write)
+Reads → Old (primary) + New (shadow — compare results)
+Monitor divergence for N days
+Switch reads to New
+Stop writes to Old
+```
+
+**Use when:** Data integrity is critical, migrating billing, financial systems, healthcare records.
+
+**Exit criteria:** Zero divergence for 7+ consecutive days across all operation types.
+
+### Blue/green deployment
+
+Two identical environments. Switch load balancer. Instant rollback.
+
+```
+LB → Blue (current production)
+     Green (new version, fully ready)
+Switch: LB → Green
+If issues: LB → Blue (rollback in <1 min)
+```
+
+**Use when:** Application migrations, not data migrations. Great for infrastructure changes.
+
+**Not ideal for:** Schema migrations — Blue and Green share the same DB, so schema must be backward compatible.
+
+## Database migration patterns
+
+### Expand-contract (zero downtime schema changes)
+
+Never alter columns in production — always expand then contract:
+
+```
+Step 1: EXPAND — add the new structure alongside the old
+  ALTER TABLE users ADD COLUMN email_v2 VARCHAR(255);
+
+Step 2: MIGRATE — dual write during deploy
+  App writes to both email and email_v2
+
+Step 3: BACKFILL — populate the new column
+  UPDATE users SET email_v2 = email WHERE email_v2 IS NULL;
+
+Step 4: SWITCH — read from new column
+  Deploy reads from email_v2
+
+Step 5: CONTRACT — remove old column
+  ALTER TABLE users DROP COLUMN email;
+
+Step 6: RENAME (if needed)
+  ALTER TABLE users RENAME COLUMN email_v2 TO email;
+```
+
+Each step is a separate deployment. Steps 1-3 are backward compatible.
+
+### Additive-only migrations
+
+Never break existing code with a migration:
+
+| Safe ✓                | Unsafe ✗                                     |
+| --------------------- | -------------------------------------------- |
+| Add nullable column   | Drop column                                  |
+| Add new table         | Rename column                                |
+| Add index             | Change column type (without expand-contract) |
+| Add nullable FK       | Add NOT NULL without default                 |
+| Rename via new column | Remove FK that code depends on               |
+
+### Large table migrations
+
+For tables > 1M rows, never run migrations in a transaction that locks the table:
+
+```sql
+-- BAD: locks table, causes downtime
+ALTER TABLE orders ADD COLUMN processed_at TIMESTAMP;
+
+-- GOOD: add nullable, backfill in batches, add NOT NULL constraint later
+ALTER TABLE orders ADD COLUMN processed_at TIMESTAMP NULL;
+
+-- Then backfill in batches:
+UPDATE orders SET processed_at = created_at
+  WHERE id BETWEEN 1 AND 10000 AND processed_at IS NULL;
+-- Repeat in 10k-row chunks with sleep between
+```
+
+Tools: `pt-online-schema-change` (MySQL), `pg_repack` (PostgreSQL), `gh-ost` (GitHub's MySQL tool).
+
+## Rollback strategies
+
+Every migration needs a rollback plan written BEFORE the migration runs.
+
+| Migration type           | Rollback approach                   |
+| ------------------------ | ----------------------------------- |
+| Schema: added column     | DROP COLUMN (if no data yet)        |
+| Schema: dropped column   | Restore from backup (can't undo)    |
+| Data: transformed values | Restore from point-in-time snapshot |
+| Code: strangler fig      | Flip feature flag to 0%             |
+| Deployment: blue/green   | Switch LB back to blue              |
+| Config: environment vars | Revert in secrets manager           |
+
+**Rule:** If a migration doesn't have a rollback plan, it's not ready to run.
+
+## Migration runbook template
+
+```markdown
+## Migration: Add `user_timezone` to profiles table
+
+**Date:** 2026-03-15 10:00 UTC
+**Owner:** @engineer
+**Estimated duration:** 25 minutes
+**Rollback time:** < 5 minutes
+
+### Pre-migration checks
+
+- [ ] Full DB backup completed (verify at: <link>)
+- [ ] Monitoring dashboards open
+- [ ] Rollback script tested in staging
+
+### Steps
+
+1. `psql $DATABASE_URL -f migrations/0042_add_user_timezone.sql` (5 min)
+2. Deploy app v2.4.1 — includes dual-write for timezone (10 min)
+3. Verify: `SELECT COUNT(*) FROM profiles WHERE timezone IS NULL` — expect < 5%
+4. Run backfill: `node scripts/backfill-timezone.js` (10 min)
+5. Verify: `SELECT COUNT(*) FROM profiles WHERE timezone IS NULL` — expect 0
+
+### Rollback
+
+If any step fails:
+
+1. `psql $DATABASE_URL -c "ALTER TABLE profiles DROP COLUMN timezone"`
+2. Redeploy v2.4.0
+3. Page on-call if data inconsistency detected
+```
+
+## Data integrity checks
+
+Before and after every migration:
+
+```sql
+-- Row counts by status (should be preserved)
+SELECT status, COUNT(*) FROM orders GROUP BY status;
+
+-- Sum of financial values (must not change)
+SELECT SUM(amount) FROM transactions;
+
+-- Check for nulls in NOT NULL columns
+SELECT COUNT(*) FROM users WHERE email IS NULL;
+
+-- Spot check sample rows
+SELECT * FROM users ORDER BY RANDOM() LIMIT 10;
+```
+
+Save pre-migration counts. Verify post-migration counts match.

package/content/plugins/migration/skills/agileflow-migration/references/rollback-playbook.md

@@ -0,0 +1,171 @@
+# Rollback Playbook
+
+**Load this when:** Planning a migration rollback, responding to a failed migration, or writing a post-mortem.
+
+## Rollback Decision Tree
+
+```
+Is data corruption or data loss occurring?
+  YES → STOP migration immediately. Initiate rollback NOW. Skip all other checks.
+  NO  ↓
+
+Is a critical user-facing service down or severely degraded?
+  YES → Rollback within 15 minutes if no hotfix available.
+  NO  ↓
+
+Is the error rate >2x pre-migration baseline AND rising?
+  YES → Rollback if no fix identified within 30 minutes.
+  NO  ↓
+
+Is the migration >50% complete with no data integrity issues?
+  YES → Strongly prefer forward-fix over rollback (rollback cost exceeds benefit).
+  NO  ↓
+
+Rollback is an option — evaluate cost vs. benefit.
+```
+
+---
+
+## Rollback Triggers by Migration Type
+
+### Schema migration
+
+| Trigger                        | Action                                   |
+| ------------------------------ | ---------------------------------------- |
+| Query errors on new schema     | Immediate rollback                       |
+| Constraint violations in prod  | Immediate rollback                       |
+| Performance regression >5x     | Rollback if not resolvable in 1h         |
+| Application errors post-deploy | Check if schema-related; rollback if yes |
+
+### Data migration
+
+| Trigger                                | Action                               |
+| -------------------------------------- | ------------------------------------ |
+| Row count mismatch vs. expected        | STOP. Investigate before proceeding. |
+| Checksum / hash validation fail        | STOP. Do not proceed.                |
+| Referential integrity errors           | Rollback source data changes         |
+| Business logic producing wrong outputs | Rollback and fix migration script    |
+
+### Dependency / library migration
+
+| Trigger                          | Action                               |
+| -------------------------------- | ------------------------------------ |
+| Test suite failure rate >10%     | Rollback package version             |
+| Runtime errors in new dependency | Rollback to previous version         |
+| Build failure                    | Rollback, investigate, fix in branch |
+
+---
+
+## Rollback Methods by Layer
+
+| Layer                  | Rollback method                          | Speed           | Data risk      |
+| ---------------------- | ---------------------------------------- | --------------- | -------------- |
+| Feature flag           | Toggle off                               | Instant         | None           |
+| Application code       | Revert deploy                            | 2–5 min         | None           |
+| Schema (additive only) | Drop added columns/tables                | Minutes         | None           |
+| Schema (destructive)   | Restore from pre-migration snapshot      | 30 min – hours  | Potential loss |
+| Data migration         | Restore from backup / run inverse script | Minutes – hours | Potential loss |
+| Infrastructure         | Terraform rollback / restore snapshot    | Minutes         | Low–medium     |
+
+---
+
+## Pre-Migration Rollback Preparation Checklist
+
+Complete before every significant migration:
+
+- [ ] Full database backup taken and verified restorable
+- [ ] Rollback script written and tested in staging
+- [ ] Feature flag created to disable new behavior without code deploy
+- [ ] Rollback decision owner named (who has authority to call it)
+- [ ] Communication plan drafted (what to say, to whom, via which channel)
+- [ ] Monitoring dashboards ready for key metrics
+- [ ] Estimated rollback time documented
+- [ ] All affected services identified (who else do we notify?)
+
+---
+
+## Communication Templates
+
+### Migration incident (initial)
+
+```
+[MIGRATION INCIDENT] {migration name} — {severity}
+
+Status: Rollback in progress / Investigating
+Impact: {what's affected — service, users, data}
+Started: {time}
+Action: {what we're doing right now}
+
+Updates every 15 minutes in #{incident-channel}.
+DRI: {name}
+```
+
+### Rollback complete
+
+```
+[RESOLVED] {migration name} — rollback complete
+
+Rolled back to: v{version} / state as of {timestamp}
+Impact duration: {start} → {end}
+Affected: {scope}
+Root cause (preliminary): {1 sentence}
+
+Post-mortem scheduled: {date/time}
+```
+
+---
+
+## Post-Mortem Structure
+
+```markdown
+# Post-Mortem: {Migration Name}
+
+**Date:** YYYY-MM-DD
+**Severity:** P1 / P2 / P3
+**Duration:** {start} → {resolved}
+**Impact:** {affected users / data volume / downtime}
+
+## Timeline
+
+| Time  | Event                  |
+| ----- | ---------------------- |
+| HH:MM | Migration started      |
+| HH:MM | First alert triggered  |
+| HH:MM | Rollback decision made |
+| HH:MM | Rollback complete      |
+
+## Root Cause
+
+[What actually went wrong]
+
+## Contributing Factors
+
+- [Factor 1]
+- [Factor 2]
+
+## What Went Well
+
+- [Thing 1]
+
+## Action Items
+
+| Action | Owner | Due |
+| ------ | ----- | --- |
+|        |       |     |
+
+## Prevention
+
+[How do we ensure this doesn't happen again?]
+```
+
+---
+
+## Rollback Risk by Migration Completeness
+
+| % Complete | Rollback recommendation                             |
+| ---------- | --------------------------------------------------- |
+| 0–20%      | Easy — rollback preferred if issues found           |
+| 20–50%     | Evaluate — rollback cost growing                    |
+| 50–80%     | Risky — forward-fix preferred unless data corrupted |
+| 80–99%     | Very risky — forward-fix strongly preferred         |
+| 100%       | Rollback is a new migration — plan carefully        |