@harness-engineering/cli 1.24.0 → 1.24.3

package/dist/agents/skills/cursor/db-expand-contract/SKILL.md

@@ -0,0 +1,199 @@
+# Expand-Contract Migration
+
+> Add new structure, migrate data, remove old structure -- the three-phase pattern for safe column renames, type changes, and table restructuring.
+
+## When to Use
+
+- Renaming a column or table in a production database
+- Changing a column type (e.g., INT to BIGINT, VARCHAR to JSONB)
+- Splitting a table or merging two tables
+- Any schema change where old and new application versions must coexist
+- Deploying backwards-compatible database changes in a rolling deployment
+
+## Instructions
+
+### Key Concepts
+
+**1. The Three Phases**
+
+- **Expand:** Add the new column/table alongside the old one. Both exist simultaneously.
+- **Migrate:** Copy data from old to new. Application writes to both (dual-write) or a trigger synchronizes them.
+- **Contract:** Remove the old column/table after all consumers use the new one.
+
+Each phase is a separate deployment. The application must work correctly at every intermediate state.
+
+**2. Column Rename Pattern**
+
+You cannot safely rename a column in a single step because the old application code still references the old name.
+
+```sql
+-- Phase 1: EXPAND -- add new column
+ALTER TABLE users ADD COLUMN full_name TEXT;
+
+-- Phase 2: MIGRATE -- backfill and dual-write
+UPDATE users SET full_name = name WHERE full_name IS NULL;
+-- Application code writes to BOTH name and full_name
+
+-- Phase 3: CONTRACT -- drop old column
+-- Only after all application instances use full_name
+ALTER TABLE users DROP COLUMN name;
+```
+
+**3. Dual-Write Strategy**
+
+During the migrate phase, the application writes to both old and new columns. This ensures either version of the application can read correctly.
+
+```sql
+-- Application INSERT during migrate phase:
+INSERT INTO users (name, full_name, email)
+VALUES ('Alice', 'Alice', 'alice@example.com');
+
+-- Trigger-based alternative (keeps sync automatic):
+CREATE OR REPLACE FUNCTION sync_name_to_full_name()
+RETURNS TRIGGER AS $$
+BEGIN
+  IF NEW.full_name IS NULL THEN
+    NEW.full_name := NEW.name;
+  END IF;
+  RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER trg_sync_name
+  BEFORE INSERT OR UPDATE ON users
+  FOR EACH ROW EXECUTE FUNCTION sync_name_to_full_name();
+```
+
+**4. Type Change Pattern**
+
+Changing a column type (e.g., `INT` to `BIGINT`) requires a full table rewrite if done directly. Expand-contract avoids this:
+
+```sql
+-- Phase 1: EXPAND
+ALTER TABLE orders ADD COLUMN order_number_v2 BIGINT;
+
+-- Phase 2: MIGRATE
+-- Backfill in batches (see db-zero-downtime-migration for batch pattern)
+UPDATE orders SET order_number_v2 = order_number
+  WHERE id BETWEEN 1 AND 10000 AND order_number_v2 IS NULL;
+-- ... repeat in batches
+
+-- Application starts reading from order_number_v2,
+-- writing to both columns
+
+-- Phase 3: CONTRACT
+ALTER TABLE orders DROP COLUMN order_number;
+ALTER TABLE orders RENAME COLUMN order_number_v2 TO order_number;
+```
+
+### Worked Example
+
+Scenario: a multi-service system needs to rename the `users.name` column to `users.display_name` while three services query the table.
+
+```sql
+-- Deploy 1: EXPAND
+ALTER TABLE users ADD COLUMN display_name TEXT;
+CREATE OR REPLACE FUNCTION sync_user_name() RETURNS TRIGGER AS $$
+BEGIN
+  IF TG_OP = 'INSERT' OR TG_OP = 'UPDATE' THEN
+    IF NEW.display_name IS NULL THEN
+      NEW.display_name := NEW.name;
+    ELSIF NEW.name IS NULL OR NEW.name != NEW.display_name THEN
+      NEW.name := NEW.display_name;
+    END IF;
+  END IF;
+  RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+CREATE TRIGGER trg_sync_user_name
+  BEFORE INSERT OR UPDATE ON users
+  FOR EACH ROW EXECUTE FUNCTION sync_user_name();
+
+-- Deploy 2: BACKFILL
+-- Run during off-peak in 5K-row batches
+UPDATE users SET display_name = name
+  WHERE display_name IS NULL
+    AND id BETWEEN $1 AND $2;
+
+-- Deploy 3-5: Update each service to read/write display_name
+-- Each service deploys independently
+
+-- Deploy 6: CONTRACT
+-- Verify: SELECT count(*) FROM users WHERE display_name IS NULL;
+-- Must return 0
+DROP TRIGGER trg_sync_user_name ON users;
+DROP FUNCTION sync_user_name();
+ALTER TABLE users DROP COLUMN name;
+```
+
+Timeline: 2-4 weeks, depending on service deployment cadence. The database is safe at every intermediate state.
+
+### Anti-Patterns
+
+1. **Renaming a column directly with `ALTER TABLE RENAME COLUMN`.** This breaks all application code referencing the old name instantly. Expand-contract avoids this by maintaining both names during transition.
+
+2. **Skipping the dual-write phase.** If old application instances still write to the old column while new instances read from the new column, data is silently lost. Both columns must stay synchronized.
+
+3. **Never completing the contract phase.** Dead columns accumulate, confuse developers, waste storage, and slow queries. Set a deadline for the contract phase and track it as tech debt.
+
+4. **Backfilling in a single transaction.** A single `UPDATE users SET display_name = name` locks millions of rows. Batch the backfill with explicit bounds and commits between batches.
+
+5. **Using database views as a permanent compatibility layer.** Views add indirection and query complexity. They are a reasonable temporary bridge during migration but should not persist after the contract phase.
+
+### PostgreSQL Specifics
+
+- `ALTER TABLE ... ADD COLUMN` is instant (no table rewrite) in PostgreSQL 11+ even with a default value.
+- Triggers used for dual-write synchronization fire inside the same transaction as the triggering statement, ensuring ACID guarantees.
+- `DROP COLUMN` does not physically remove data; it marks the column as dropped. Use `VACUUM FULL` or `pg_repack` to reclaim space afterward if needed.
+- PostgreSQL supports transactional DDL, so the expand phase can be wrapped in a transaction for atomicity.
+
+## Details
+
+### Advanced Topics
+
+**Table Splits:** When splitting a wide table into two narrower tables (vertical split), expand-contract applies at the table level:
+
+1. Create the new table with a foreign key to the original
+2. Backfill data from the original table
+3. Update application queries to join or query the new table
+4. Drop the migrated columns from the original table
+
+**Feature Flags for Contract Phase:** Use application-level feature flags to control which column the application reads from. This makes the contract phase reversible: if issues arise, flip the flag back to the old column.
+
+**Automated Expand-Contract:** Tools like `reshape` (Rust-based schema migration) automate the expand-contract pattern. They generate the triggers and handle the dual-write automatically, reducing manual error.
+
+### Engine Differences
+
+MySQL lacks transactional DDL. Each `ALTER TABLE` commits implicitly, so a multi-step expand operation cannot be rolled back atomically. This makes careful planning even more critical: if the expand step partially succeeds, manual cleanup is required.
+
+MySQL `pt-online-schema-change` and `gh-ost` can automate the expand-contract pattern for column type changes by creating a shadow table, copying data, and swapping. They effectively implement expand-contract internally.
+
+### Real-World Case Studies
+
+GitHub migrated their `repositories.id` column from INT to BIGINT on a table with billions of rows. The expand-contract approach took several weeks: they added a `new_id BIGINT` column, set up dual-write synchronization via application code, backfilled the new column in background batches, updated all queries to use the new column, and finally dropped the old column. At no point was the service interrupted. The migration was invisible to users.
+
+## Source
+
+- [PostgreSQL ALTER TABLE](https://www.postgresql.org/docs/current/sql-altertable.html)
+- Sadalage, P. & Fowler, M. "Refactoring Databases" (2006)
+- [Expand-Contract Pattern](https://openpracticelibrary.com/practice/expand-and-contract-pattern/)
+
+## Process
+
+1. Add the new column or table (expand) with appropriate defaults and a sync trigger.
+2. Backfill existing data in batches; enable dual-write in application code.
+3. Deploy each consuming service to read from the new structure.
+4. Verify all data is migrated and all services are updated, then drop the old structure (contract).
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** db-zero-downtime-migration, db-migration-rollback, db-acid-properties, db-acid-in-practice
+
+## Success Criteria
+
+- Schema changes are decomposed into expand, migrate, and contract phases deployed separately.
+- Old and new columns coexist with dual-write or trigger-based synchronization during migration.
+- Data backfills run in batches with verification queries confirming completeness before the contract phase.
+- The contract phase removes the old structure only after all consumers have migrated.

package/dist/agents/skills/cursor/db-expand-contract/skill.yaml

@@ -0,0 +1,40 @@
+name: db-expand-contract
+version: '1.0.0'
+description: Add new, migrate data, remove old -- safe column and table renames using expand-contract
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - db-zero-downtime-migration
+  - db-migration-rollback
+  - db-acid-properties
+  - db-acid-in-practice
+stack_signals:
+  - postgresql
+  - mysql
+  - database
+keywords:
+  - expand-contract
+  - column-rename
+  - table-rename
+  - safe-migration
+  - backwards-compatible
+  - dual-write
+  - backfill
+metadata:
+  author: community
+  upstream: postgresql.org/docs/current/sql-altertable.html
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/cursor/db-explain-reading/SKILL.md

@@ -0,0 +1,213 @@
+# Reading EXPLAIN Output
+
+> How to read query execution plans to identify performance bottlenecks, row count misestimations, and missing indexes.
+
+## When to Use
+
+- Diagnosing slow queries in development or production
+- Verifying that indexes are being used as expected
+- Comparing alternative query formulations
+- Validating performance impact of schema changes
+- Investigating performance regressions after deployments or data migrations
+
+## Instructions
+
+### Key Concepts
+
+**EXPLAIN** shows the query plan without executing the query. **EXPLAIN ANALYZE** executes the query and shows actual timing and row counts alongside estimates.
+
+```sql
+-- Plan only (no execution):
+EXPLAIN SELECT * FROM orders WHERE status = 'active';
+
+-- Plan with actual execution metrics:
+EXPLAIN ANALYZE SELECT * FROM orders WHERE status = 'active';
+
+-- Full diagnostic output (recommended):
+EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
+SELECT * FROM orders WHERE status = 'active';
+```
+
+**Key fields in EXPLAIN output:**
+
+- `cost=startup..total` -- estimated cost in arbitrary units (based on `seq_page_cost`). Startup cost is the work before the first row is returned. Total cost is the work for all rows.
+- `rows` -- estimated number of rows the node will produce
+- `width` -- estimated average row width in bytes
+
+**Additional fields with ANALYZE:**
+
+- `actual time=startup..total` -- real milliseconds
+- `rows` -- actual number of rows produced
+- `loops` -- how many times this node executed (important for nested loops)
+
+**Reading order:** Plans read bottom-up and inside-out. The deepest-indented node executes first. Each parent node consumes the output of its children.
+
+**BUFFERS** adds I/O detail:
+
+- `shared hit` -- pages found in the buffer cache (fast)
+- `shared read` -- pages read from disk (slow)
+
+### Worked Example
+
+A JOIN query with performance issues:
+
+```sql
+EXPLAIN (ANALYZE, BUFFERS)
+SELECT o.id, o.total, c.name
+FROM orders o
+JOIN customers c ON c.id = o.customer_id
+WHERE o.status = 'pending' AND c.region = 'us-east';
+```
+
+```
+Nested Loop (cost=1.00..45231.21 rows=10 width=28)
+           (actual time=0.082..4521.340 rows=48500 loops=1)
+  Buffers: shared hit=12340 read=89012
+  ->  Index Scan using idx_orders_status on orders o
+        (cost=0.43..8234.12 rows=10 width=16)
+        (actual time=0.031..234.120 rows=48500 loops=1)
+        Index Cond: (status = 'pending')
+        Buffers: shared hit=5234 read=3421
+  ->  Index Scan using customers_pkey on customers c
+        (cost=0.29..3.71 rows=1 width=20)
+        (actual time=0.085..0.086 rows=1 loops=48500)
+        Index Cond: (id = o.customer_id)
+        Filter: (region = 'us-east')
+        Rows Removed by Filter: 0
+        Buffers: shared hit=7106 read=85591
+Planning Time: 0.342 ms
+Execution Time: 4523.891 ms
+```
+
+**Diagnosis:**
+
+1. The planner estimated 10 rows from orders but got 48,500 -- a **4,850x misestimation**
+2. Because it expected 10 rows, it chose Nested Loop (good for few rows) instead of Hash Join (better for many rows)
+3. The inner loop executes 48,500 times (`loops=48500`), each doing an index scan on customers
+4. `shared read=85591` on customers shows heavy disk I/O
+
+**Fix:** Update statistics to correct the misestimation:
+
+```sql
+ANALYZE orders;
+```
+
+After ANALYZE, the planner sees the correct row count and switches to Hash Join:
+
+```
+Hash Join (cost=1234.56..9876.54 rows=48200 width=28)
+          (actual time=12.340..89.120 rows=48500 loops=1)
+  Buffers: shared hit=8923 read=1234
+Execution Time: 91.234 ms
+```
+
+Query time dropped from 4.5 seconds to 91 milliseconds.
+
+### Anti-Patterns
+
+1. **Using EXPLAIN without ANALYZE.** You only see estimated costs and rows -- not reality. Always use ANALYZE for performance diagnosis (but see the next point for destructive queries).
+
+2. **Running EXPLAIN ANALYZE on destructive queries without a transaction.**
+
+```sql
+-- WRONG: actually deletes rows
+EXPLAIN ANALYZE DELETE FROM orders WHERE status = 'cancelled';
+
+-- CORRECT: analyze without side effects
+BEGIN;
+EXPLAIN ANALYZE DELETE FROM orders WHERE status = 'cancelled';
+ROLLBACK;
+```
+
+3. **Ignoring the rows discrepancy.** When estimated rows differ from actual rows by more than 10x, the planner likely chose a suboptimal strategy. Fix statistics with `ANALYZE tablename;`.
+
+4. **Focusing only on total cost.** A node with low total cost but high `loops` can dominate execution time. Multiply `actual time` by `loops` to get the true cost of a node.
+
+### PostgreSQL Specifics
+
+**JSON format for programmatic parsing:**
+
+```sql
+EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) SELECT ...;
+```
+
+**auto_explain extension** logs slow query plans automatically:
+
+```sql
+-- In postgresql.conf:
+-- shared_preload_libraries = 'auto_explain'
+-- auto_explain.log_min_duration = '1s'
+```
+
+**EXPLAIN (ANALYZE, BUFFERS, WAL)** in PostgreSQL 13+ shows WAL bytes generated -- useful for understanding write overhead.
+
+**pg_stat_statements** identifies which queries need EXPLAIN. Sort by `total_exec_time` to find the biggest offenders:
+
+```sql
+SELECT query, calls, mean_exec_time, total_exec_time
+FROM pg_stat_statements
+ORDER BY total_exec_time DESC
+LIMIT 10;
+```
+
+## Details
+
+### Advanced Topics
+
+**JIT compilation indicators.** PostgreSQL 11+ may show JIT compilation in EXPLAIN output for complex queries. `JIT: Functions: N, Generation Time: Xms, Inlining Time: Xms`. JIT helps for CPU-bound queries but adds planning overhead.
+
+**Parallel query plans.** Look for `Workers Planned: N` and `Workers Launched: N`. If launched < planned, the system ran out of parallel workers (`max_parallel_workers_per_gather`).
+
+**CTE Scan nodes.** Before PostgreSQL 12, CTEs were always materialized (written to a temp tuplestore). The CTE Scan node reads from this materialized result. In PostgreSQL 12+, CTEs may be inlined -- no CTE Scan node appears.
+
+**SubPlan vs InitPlan.** SubPlan executes once per outer row (correlated subquery). InitPlan executes once for the entire query (uncorrelated subquery). SubPlan with high outer row counts is a performance red flag.
+
+**"Never executed" nodes.** In conditional plans (e.g., with `CASE` or parameterized bitmap scans), some nodes show `(never executed)`. This is normal -- the runtime condition was not met.
+
+### Engine Differences
+
+**MySQL EXPLAIN** uses a tabular format with different columns:
+
+| MySQL Column    | PostgreSQL Equivalent         |
+| --------------- | ----------------------------- |
+| `type`          | Node type (ALL, index, range) |
+| `possible_keys` | Candidate indexes             |
+| `key`           | Chosen index                  |
+| `rows`          | Estimated rows                |
+| `filtered`      | Selectivity percentage        |
+| `Extra`         | Additional info (Using index) |
+
+**MySQL EXPLAIN ANALYZE** (8.0.18+) shows actual timing with tree format:
+
+```sql
+EXPLAIN ANALYZE SELECT * FROM orders WHERE status = 'active';
+```
+
+MySQL `EXPLAIN FORMAT=TREE` is the closest equivalent to PostgreSQL's default text output. MySQL lacks a BUFFERS equivalent -- I/O diagnostics require Performance Schema or `SHOW STATUS` counters.
+
+### Real-World Case Studies
+
+**API endpoint taking 8 seconds.** EXPLAIN ANALYZE on the slow query revealed a Nested Loop join with `rows=10` estimated but `rows=200000` actual. The stale statistics caused the planner to choose Nested Loop instead of Hash Join. Running `ANALYZE orders;` updated the statistics, the planner switched to Hash Join, and the query dropped from 8 seconds to 50ms. The fix was added to the CI pipeline: `ANALYZE` runs after every migration.
+
+## Source
+
+- [PostgreSQL Using EXPLAIN](https://www.postgresql.org/docs/current/using-explain.html)
+- [pgMustard -- EXPLAIN Analysis Tool](https://www.pgmustard.com/)
+
+## Process
+
+1. Read the key concepts to understand EXPLAIN output fields and reading order.
+2. Apply EXPLAIN (ANALYZE, BUFFERS) to slow queries, comparing estimated vs actual rows and identifying I/O bottlenecks.
+3. Verify fixes by re-running EXPLAIN ANALYZE after adding indexes, updating statistics, or rewriting queries.
+
+## Harness Integration
+
+- **Type:** knowledge -- this skill is a reference document, not a procedural workflow.
+- **No tools or state** -- consumed as context by other skills and agents.
+- **related_skills:** db-scan-types, db-query-statistics, db-query-rewriting, db-btree-index, db-composite-index, db-covering-index
+
+## Success Criteria
+
+- EXPLAIN ANALYZE output is interpreted correctly with estimated vs actual rows compared.
+- Row count misestimations greater than 10x are identified and resolved with ANALYZE.
+- Destructive queries are always wrapped in transactions before running EXPLAIN ANALYZE.

package/dist/agents/skills/cursor/db-explain-reading/skill.yaml

@@ -0,0 +1,44 @@
+name: db-explain-reading
+version: '1.0.0'
+description: Reading EXPLAIN and EXPLAIN ANALYZE output, understanding cost estimation, and comparing actual vs estimated rows
+stability: static
+cognitive_mode: advisory-guide
+type: knowledge
+tier: 3
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+  - cursor
+  - codex
+tools: []
+paths: []
+related_skills:
+  - db-scan-types
+  - db-query-statistics
+  - db-query-rewriting
+  - db-btree-index
+  - db-composite-index
+  - db-covering-index
+  - prisma-performance-patterns
+  - drizzle-performance-patterns
+stack_signals:
+  - postgresql
+  - mysql
+  - database
+keywords:
+  - EXPLAIN
+  - EXPLAIN-ANALYZE
+  - query-plan
+  - cost-estimation
+  - actual-rows
+  - execution-time
+  - planner
+metadata:
+  author: community
+  upstream: postgresql.org/docs/current/using-explain.html
+state:
+  persistent: false
+  files: []
+depends_on: []