@coralai/sps-cli 0.42.0 → 0.44.0

package/skills/coding-standards/references/tdd.md

@@ -0,0 +1,171 @@
+# TDD — Test-Driven Development
+
+The cycle that keeps behavior ahead of implementation. Language-neutral.
+
+## The cycle
+
+1. **RED** — write the smallest failing test for the next bit of behavior.
+2. **GREEN** — write the minimum code that makes it pass. Ugly is fine.
+3. **REFACTOR** — clean up while keeping all tests green.
+
+Repeat. Each loop should take minutes, not hours.
+
+```
+┌─────────┐     ┌──────────┐     ┌──────────┐
+│  RED    │────▶│  GREEN   │────▶│ REFACTOR │──┐
+│ (fail)  │     │ (pass)   │     │ (still   │  │
+└─────────┘     └──────────┘     │  pass)   │  │
+     ▲                           └──────────┘  │
+     └─────────────────────────────────────────┘
+```
+
+## Why bother
+
+- **You write less code.** You only write what a test demands.
+- **You design the interface first.** Tests force you to use your API before anyone else does.
+- **Regressions get names.** Every bug fix starts with a failing test that reproduces it.
+- **Coverage is real.** Not a number chased after the fact.
+
+TDD isn't about testing. It's about designing by answering "how would I use this?" before you build it.
+
+## When TDD shines
+
+- New business logic with clear inputs/outputs
+- Fixing a bug (always: reproduce with a test first, then fix)
+- Pure functions, state machines, parsers
+- Rules-heavy code (pricing, validation, authz)
+
+## When TDD is awkward
+
+- UI polish (use snapshot / visual tests instead)
+- Exploratory spikes (throw the code away; tests would be premature)
+- Integration against an unknown third-party API (explore, then test)
+- Glue code with no logic (one thin integration test is enough)
+
+Awkward ≠ exempt. You still verify. You just choose the right tool.
+
+## Writing the first test
+
+Think: what's the smallest visible behavior?
+
+```
+# Feature: "sum two numbers"
+
+# RED (smallest failing test)
+assert add(2, 3) == 5
+
+# GREEN (smallest passing code)
+def add(a, b):
+    return a + b
+
+# Now push the next edge case
+assert add(-1, 1) == 0        # pass already — good
+assert add(0.1, 0.2) == 0.3   # FAILS (float math) — now you've learned something real
+```
+
+If the first test passes without you writing code, either the feature exists, the test is vacuous, or you're testing the wrong thing. Stop and rethink.
+
+## One assertion per behavior, not per test
+
+A single test can have several assertions if they verify the same behavior from different angles:
+
+```
+# OK — all verifying "create returns a persisted user"
+u = service.create(name="A", email="a@x.com")
+assert u.id is not None
+assert u.name == "A"
+assert u.email == "a@x.com"
+assert db.find(u.id) == u
+```
+
+Not OK:
+
+```
+# Two behaviors in one test
+u = service.create(...)
+assert u.id is not None          # behavior 1: create
+service.delete(u.id)
+assert db.find(u.id) is None     # behavior 2: delete  ← split
+```
+
+One test fails → one problem. Mixing behaviors makes failures ambiguous.
+
+## Reproducing a bug
+
+Every bug fix starts with a test that fails *because of the bug*.
+
+```
+# 1. Write the failing test (RED)
+def test_negative_balance_rejected():
+    with pytest.raises(ValidationError):
+        account.withdraw(999_999)
+
+# 2. Fix the code (GREEN)
+
+# 3. Commit both in the same change
+```
+
+Without the test, the bug can silently come back. With it, the regression is guaranteed to be caught.
+
+## What a good test looks like
+
+| Property | Why |
+|---|---|
+| **Fast** | You run it often; slow tests get skipped |
+| **Isolated** | No order dependency; no shared mutable state |
+| **Repeatable** | Same result every run; no flakes |
+| **Self-verifying** | Pass/fail is automatic, not a human reading output |
+| **Named for behavior** | `test_rejects_empty_email`, not `test_1` |
+
+Flaky tests are worse than missing tests — they train you to ignore red.
+
+## Test pyramid
+
+```
+            ▲
+            │      E2E              ← few, slow, high-value
+            │    (5%)
+            │  Integration          ← some, medium speed
+            │    (20%)
+            │  Unit                 ← many, fast
+            │    (75%)
+            ▼
+```
+
+Invert this (lots of E2E, no unit) and you get slow CI, brittle tests, and bugs that hide in the unit level. Unit tests are the backbone; E2E is the sanity check.
+
+## Refactor only on green
+
+Rule: never refactor with failing tests. You won't know whether the refactor broke something or the test was already broken.
+
+Steps:
+1. Get to green.
+2. Commit.
+3. Refactor.
+4. Stay green.
+5. Commit.
+
+Small commits. Rollback is free.
+
+## Coverage targets (sane defaults)
+
+| Layer | Target |
+|---|---|
+| Pure / domain logic | ≥ 90% |
+| Application / use cases | ≥ 80% |
+| Adapters (DB, HTTP) | ≥ 60%; integration tests carry the rest |
+| Glue (bootstrap, composition) | ≥ 0% — don't chase coverage here |
+
+Coverage is a floor, not a ceiling. 100% coverage with weak assertions is still untested code.
+
+## Anti-patterns
+
+| Anti-pattern | Fix |
+|---|---|
+| Writing tests after the code "to hit coverage" | That's not TDD; and coverage isn't the goal |
+| One mega-test per feature | Split by behavior |
+| Testing mocks (the mock returns X, assert it returned X) | Test the thing, or delete the test |
+| Asserting on implementation detail (private method called) | Assert on observable behavior |
+| Tests that sleep to wait for async | Use deterministic scheduling / fake clocks |
+| Disabling a failing test to unblock CI | Fix it or delete it. Disabled tests lie. |
+| Fixture soup — hundreds of lines to set up a test | Your system is hard to use; that's the signal, not the fixture |

package/skills/database/SKILL.md

@@ -0,0 +1,53 @@
+---
+name: database
+description: Database end skill — schema design, indexing, queries, migrations, scaling. Engine-neutral (Postgres / MySQL / SQLite / NoSQL patterns). Pair with a language skill (`python`, `typescript`, `golang`), `backend` for where it fits, and `coding-standards`.
+origin: ecc-fork + original (https://github.com/affaan-m/everything-claude-code, MIT)
+---
+
+# Database
+
+Schema design, indexing, queries, migrations, scaling. **Engine-neutral**. For query-level performance inside a service, also see `backend/references/data-access.md`.
+
+## When to load
+
+- Designing or reviewing schema changes
+- Writing migrations
+- Diagnosing slow queries / choosing indexes
+- Planning multi-tenant / sharding / read-replica topology
+- Choosing between relational, document, key-value, time-series, graph
+
+## Core principles
+
+1. **Model the domain, not the queries.** Normalized schema first; denormalize when measurement shows it's needed.
+2. **Every table has a primary key.** Every foreign key is declared, so integrity is enforced by the DB, not hoped for in app code.
+3. **Constraints over application validation for invariants.** `NOT NULL`, `CHECK`, `UNIQUE`, `FOREIGN KEY` — let the DB refuse bad data.
+4. **Indexes are a cost.** Every index slows writes and costs storage. Add them to answer real queries; drop them when the queries change.
+5. **Migrations are additive first, destructive later.** Deploy the read; deploy the write; backfill; then remove the old path.
+6. **Test migrations against production-scale data.** A migration that runs in 5 seconds on 1 000 rows can lock a table for 30 minutes on 100 million.
+7. **Keep transactions short.** Long transactions hold locks that block everyone else.
+8. **Pick the right tool.** Postgres gets you far — but not everything is a relational problem.
+
+## How to use references
+
+| Reference | When to load |
+|---|---|
+| [`references/schema.md`](references/schema.md) | Normalization, keys, constraints, types, partitioning, soft vs. hard delete |
+| [`references/indexing.md`](references/indexing.md) | B-tree, partial, expression, multi-column, GIN / GiST, covering indexes |
+| [`references/queries.md`](references/queries.md) | JOINs, subqueries, CTEs, window functions, EXPLAIN |
+| [`references/migrations.md`](references/migrations.md) | Zero-downtime migrations, rename columns, schema evolution, backfills |
+| [`references/scaling.md`](references/scaling.md) | Replication, sharding, caching, pooling, partitioning |
+| [`references/nosql.md`](references/nosql.md) | Document, key-value, time-series, graph — when each fits |
+
+## Forbidden patterns (auto-reject)
+
+- Tables without a primary key
+- Foreign keys without a declared reference (rely-on-app-code pattern)
+- `NULL` meaning two different things (e.g. "unknown" vs "not applicable")
+- Storing JSON blobs as the primary way to represent structured data (use columns for what you query)
+- `SELECT *` in production application code
+- Indexes added without a query that uses them
+- Destructive migrations without a tested rollback plan
+- Running `ALTER TABLE` on a large table during peak hours without locking analysis
+- Business logic in stored procedures that the code doesn't see
+- `TIMESTAMP` without timezone where the app runs in multiple regions
+- Money stored as `FLOAT` / `DOUBLE` — use `DECIMAL` / cents-as-integer

package/skills/database/references/indexing.md

@@ -0,0 +1,190 @@
+# Indexing
+
+B-tree, partial, expression, multi-column, GIN/GiST, covering. Read `EXPLAIN`, don't guess.
+
+## Why indexes matter
+
+Without an index, the DB does a sequential scan — O(n) per query. With an index, it's O(log n). On a table of 50 million rows, that's the difference between 2 ms and 20 seconds.
+
+The catch: indexes cost storage and slow writes. Every `INSERT` / `UPDATE` to an indexed column updates the index too. More indexes → more write amplification.
+
+## Default: B-tree
+
+The workhorse. Use for equality and range queries on ordered data.
+
+```sql
+CREATE INDEX ix_users_email ON users(email);
+
+-- Helps:
+SELECT * FROM users WHERE email = 'a@x.com';
+SELECT * FROM users WHERE email BETWEEN 'a' AND 'c';
+SELECT * FROM users ORDER BY email LIMIT 20;
+```
+
+## Multi-column indexes
+
+Order matters. An index on `(a, b)` helps:
+
+```sql
+WHERE a = ? AND b = ?    -- yes
+WHERE a = ?              -- yes (uses prefix)
+WHERE b = ?              -- no (would need an index starting with b)
+WHERE a = ? ORDER BY b   -- yes, range-limited
+```
+
+Rule: most-selective (highest-cardinality) column first, or the column used without the other.
+
+Don't create `ix_a_b` AND `ix_a` AND `ix_a_b_c` — the broadest index already covers the narrower prefixes. Prune.
+
+## Partial indexes
+
+Index only the rows that matter for the hot query.
+
+```sql
+CREATE INDEX ix_orders_pending ON orders(created_at)
+    WHERE status = 'pending';
+```
+
+Smaller index, faster. Ideal when the filtered subset is a small fraction of the table.
+
+## Expression indexes
+
+Index a computed value.
+
+```sql
+CREATE INDEX ix_users_email_lower ON users (LOWER(email));
+
+-- Helps:
+SELECT * FROM users WHERE LOWER(email) = 'a@x.com';
+```
+
+Useful for case-insensitive search, computed columns, JSON extraction:
+
+```sql
+CREATE INDEX ix_events_user ON events ((payload->>'user_id'));
+```
+
+## Covering / include columns
+
+Include extra columns in the index so the DB doesn't need to visit the table.
+
+```sql
+CREATE INDEX ix_orders_user_created ON orders(user_id, created_at)
+    INCLUDE (total_cents, status);
+
+-- Index-only scan:
+SELECT total_cents, status FROM orders
+    WHERE user_id = ? ORDER BY created_at LIMIT 20;
+```
+
+The query is answered entirely from the index pages — big win on wide tables.
+
+## Specialized indexes (Postgres examples; others vary)
+
+| Type | Use |
+|---|---|
+| `GIN` | JSONB fields, arrays, full-text search |
+| `GiST` | Ranges, geo, custom types |
+| `BRIN` | Very large append-only tables (logs, time-series) |
+| `Hash` | Equality on huge keys (Postgres 10+); B-tree usually wins |
+
+Example — full text:
+
+```sql
+CREATE INDEX ix_articles_tsv ON articles
+    USING GIN (to_tsvector('english', title || ' ' || body));
+```
+
+## When to add an index
+
+Start with the query that's slow. Run `EXPLAIN (ANALYZE, BUFFERS)` on it.
+
+```
+Seq Scan on orders  (cost=0.00..25000.00 rows=1000 width=48)
+  Filter: (status = 'pending')
+  Rows Removed by Filter: 499000
+```
+
+`Seq Scan` on a filter that returned 1 / 500 rows is wasteful — add an index. After:
+
+```
+Index Scan using ix_orders_pending  (cost=0.42..8.44 rows=1)
+```
+
+Don't add indexes speculatively. Adding an index "in case someone queries by this" creates write cost for zero read benefit.
+
+## When NOT to add an index
+
+- Column is low-cardinality (`gender` with 3 values, `status` with 4 values on a 10-million table) — DB will prefer sequential scan. Exception: partial index on the rare value.
+- Table is small (< ~10 000 rows). The optimizer picks seq scan anyway.
+- Column is frequently updated and rarely queried. Write cost > read savings.
+- Column is already covered by a broader index prefix.
+
+## Check what's being used
+
+Postgres:
+
+```sql
+SELECT indexname, idx_scan, idx_tup_read, idx_tup_fetch
+FROM pg_stat_user_indexes
+ORDER BY idx_scan ASC;
+```
+
+Indexes with `idx_scan = 0` for weeks are candidates for drop.
+
+## Maintenance
+
+- **Postgres**: `VACUUM ANALYZE` keeps statistics fresh. Autovacuum handles most cases; tune thresholds on write-heavy tables.
+- **MySQL**: `ANALYZE TABLE` periodically.
+- **Rebuilding** fragmented indexes: `REINDEX CONCURRENTLY` (Postgres 12+) — online rebuild.
+
+## Index creation on live tables
+
+Naive `CREATE INDEX` locks the table for writes. On a hot table, that stalls traffic.
+
+```sql
+-- ❌ on a hot table
+CREATE INDEX ix_x ON big_table (y);
+
+-- ✅
+CREATE INDEX CONCURRENTLY ix_x ON big_table (y);
+```
+
+`CONCURRENTLY` (Postgres) doesn't block writes. Takes longer; retry on failure (partial indexes linger as `INVALID`).
+
+MySQL 5.6+ supports online index creation for most storage engines; check the engine.
+
+## JOIN indexes
+
+For a query that joins `orders.user_id = users.id`, both sides need indexes on those columns. `users.id` is already the PK; `orders.user_id` needs its own index (declaring a FK does NOT auto-create the index in Postgres).
+
+## ORDER BY + LIMIT
+
+For paginated lists:
+
+```sql
+SELECT * FROM events
+    WHERE tenant_id = ?
+    ORDER BY created_at DESC
+    LIMIT 50;
+```
+
+Index: `(tenant_id, created_at DESC)`. With `LIMIT`, the DB reads from the index and stops at 50 — O(1) regardless of total rows.
+
+## Covering index vs. cache
+
+Sometimes the right answer is caching, not more indexes. If a query is inherently expensive (joins, aggregations), cache the result and invalidate on write. See `backend/references/caching.md`.
+
+## Anti-patterns
+
+| Anti-pattern | Fix |
+|---|---|
+| Index on every column "just in case" | Index for queries; drop unused |
+| Single-column indexes where composite would serve | One good composite > three singles |
+| Same leading column in many indexes | The broader index covers the narrower; prune |
+| Function in WHERE without an expression index | Index the expression, or rewrite the query |
+| `LIKE '%x%'` with hopes of index use | Only trailing wildcards use B-tree; use full-text for leading wildcard |
+| Non-sargable queries: `WHERE DATE(col) = ?` | Use ranges: `col >= '2026-04-20' AND col < '2026-04-21'` |
+| Indexing enum-like TEXT column with only 3 values on a hot write table | Partial index, or skip the index |
+| Blocking `CREATE INDEX` on live prod | `CONCURRENTLY` / online |
+| Forgetting to index foreign keys | Every FK column should be indexed for JOINs |

package/skills/database/references/migrations.md

@@ -0,0 +1,199 @@
+# Migrations
+
+Zero-downtime changes. Every migration survives prod traffic.
+
+## Rules
+
+1. **Every schema change is a migration file.** Checked in, versioned, applied in CI/CD.
+2. **Never edit a merged migration.** Write a new one.
+3. **Additive first, destructive later.** Add → backfill → switch → remove.
+4. **Test against production-scale data.** Time on staging ≠ time on prod.
+5. **Every migration has a rollback story.** Either a reverse migration or a forward-only fix plan.
+
+## Migration tools
+
+| Tool | Works with |
+|---|---|
+| Flyway, Liquibase | Java / any SQL |
+| Django migrations | Django |
+| Alembic | Python / SQLAlchemy |
+| Rails migrations | Rails |
+| Prisma migrate | Node / TypeScript |
+| Goose, Atlas | Go / any |
+| Sqitch | Standalone, VCS-driven |
+| Raw SQL + shell | Simple, works everywhere |
+
+Pick one per project; don't mix. All of them do the same core thing: apply ordered SQL files once, track what's applied in a meta table.
+
+## Additive vs. destructive
+
+| Additive (safe) | Destructive (careful) |
+|---|---|
+| `CREATE TABLE` | `DROP TABLE` |
+| `ADD COLUMN ... NULL` | `DROP COLUMN` |
+| `CREATE INDEX CONCURRENTLY` | `ALTER COLUMN ... TYPE` (if rewrites table) |
+| New `CHECK` (not validated yet) | `ALTER COLUMN ... NOT NULL` on a non-null column existing data |
+| New FK (not validated yet) | Renames |
+
+Pure additive migrations are (almost) always safe. Destructive ones need a phased plan.
+
+## The expand / contract pattern
+
+```
+t0: schema A       app code A
+t1: schema A + B   app code A        (expand: new schema coexists)
+t2: schema A + B   app code B        (switch: app uses B)
+t3: schema B       app code B        (contract: old schema removed)
+```
+
+Each step deployable independently. Rollback = go back one step.
+
+### Example: rename a column
+
+Don't just `ALTER TABLE users RENAME COLUMN email_address TO email`. Code that's still running reads from `email_address` and crashes.
+
+```
+Deploy 1: ADD COLUMN email TEXT;                    -- expand
+Deploy 2: backfill: UPDATE users SET email = email_address WHERE email IS NULL;
+Deploy 3: app reads/writes BOTH email and email_address (dual-write).
+Deploy 4: app reads/writes email only.
+Deploy 5: DROP COLUMN email_address;                 -- contract
+```
+
+Tedious but survives any restart sequence.
+
+### Example: adding a NOT NULL column
+
+```
+Deploy 1: ADD COLUMN role TEXT DEFAULT 'user';       -- default avoids rewrite in Postgres 11+
+Deploy 2: app writes role on all new rows.
+Deploy 3: backfill rows with NULL role.
+Deploy 4: ALTER COLUMN role SET NOT NULL.
+Deploy 5: (optional) DROP DEFAULT if you don't want it anymore.
+```
+
+Don't do `ADD COLUMN NOT NULL DEFAULT ...` on a huge table in old DB versions — it rewrites the whole table.
+
+## Index creation on live tables
+
+```sql
+-- ❌ locks the table
+CREATE INDEX ix_users_email ON users(email);
+
+-- ✅ online
+CREATE INDEX CONCURRENTLY ix_users_email ON users(email);
+```
+
+`CONCURRENTLY` runs in its own transaction (no DDL bundled). Migration frameworks that wrap every migration in a transaction will reject this — check whether your tool supports "non-transactional migrations".
+
+## Backfills
+
+For large tables, don't backfill in one statement. Batch:
+
+```sql
+UPDATE users SET role = 'user'
+WHERE id IN (
+    SELECT id FROM users
+    WHERE role IS NULL
+    ORDER BY id
+    LIMIT 10000
+);
+```
+
+Loop until no rows updated. Sleep between batches to give replicas time to catch up.
+
+Script it outside the migration framework so a slow backfill doesn't block a deploy. The schema change goes through the migration file; the backfill is a job.
+
+## Rename a table
+
+Similar expand/contract:
+
+```
+Deploy 1: CREATE TABLE new_name (... same schema ...);
+Deploy 2: app writes to both (dual-write); or create a view / sync trigger.
+Deploy 3: backfill historical rows.
+Deploy 4: app reads/writes new_name only.
+Deploy 5: DROP TABLE old_name.
+```
+
+Most teams avoid this unless strictly necessary.
+
+## Change a column type
+
+If the new type is a strict widening (`INT` → `BIGINT` in some DBs, `VARCHAR(50)` → `VARCHAR(100)`), the change is cheap and online.
+
+If it rewrites (`TEXT` → `INT` with a cast), treat as destructive + expand/contract.
+
+```
+ALTER TABLE users ALTER COLUMN x TYPE BIGINT USING x::BIGINT;
+-- Rewrites the column; blocks writes. On a big table, that hurts.
+```
+
+Alternative: add new column, backfill, switch app, drop old.
+
+## Dropping columns / tables
+
+1. App stops reading the column (deploy).
+2. App stops writing the column (deploy).
+3. Migration drops it.
+
+Skipping 1 and 2 causes crashes when old app instances hit the dropped column during a rolling deploy.
+
+## Migration performance
+
+```sql
+-- Before running in prod
+EXPLAIN ANALYZE <the migration statement>
+```
+
+Or time it on a copy of prod. A migration that takes an hour of lock time is a migration that breaks the site.
+
+Rule of thumb for Postgres:
+- `CREATE TABLE`: instant.
+- `ADD COLUMN` nullable (11+): instant (metadata only).
+- `ADD COLUMN NOT NULL DEFAULT` (11+): instant.
+- `ALTER COLUMN ... TYPE` that rewrites: `O(rows)`, blocks writes.
+- `CREATE INDEX CONCURRENTLY`: online, takes time.
+- Adding FK constraint `VALIDATED`: scans the full table. Create `NOT VALID` then `VALIDATE CONSTRAINT` to split the pain.
+
+## Transactional DDL
+
+Postgres wraps DDL in transactions. If something fails mid-way, the migration rolls back cleanly. Mix DDL + data: keep it short; long DDL transactions hold locks.
+
+MySQL pre-8 didn't support transactional DDL — failed migrations left partial state. Recovery is manual.
+
+## Rollback plans
+
+Before deploying, write down:
+- What's the rollback command?
+- What if the migration has already partly run?
+- What if the backfill ran but the schema change failed?
+
+For irreversible migrations (dropping a column), rollback = restore from backup + re-apply changes after. State that explicitly; don't pretend it's reversible.
+
+## Feature flags + schema
+
+For risky schema changes, gate the new code path behind a flag:
+
+```
+if feature('new_schema_path'):
+    use new table
+else:
+    use old table
+```
+
+Flip at will, revert instantly without a migration. Pair with expand/contract for best results.
+
+## Anti-patterns
+
+| Anti-pattern | Fix |
+|---|---|
+| Editing an applied migration | New migration |
+| Dropping a column in the same deploy as code stops using it | Stage deploys |
+| `ALTER TABLE ADD COLUMN NOT NULL DEFAULT 'x'` on a 100M-row table (old Postgres) | Split: ADD nullable → backfill → SET NOT NULL |
+| No rollback plan | Document; even "forward-only" needs a plan |
+| Running backfill in a single statement on a huge table | Batch + pause |
+| Putting `CREATE INDEX` inside a long transaction | Use `CONCURRENTLY`, separate transaction |
+| Dropping FKs "for performance" during a backfill | Disable constraints for the migration window deliberately |
+| Untested migrations on prod-like data | Run against a snapshot before production |
+| Migrations that depend on app code behaviour | Keep migrations self-contained in SQL |