@caddis/cli 0.1.0 → 0.2.0

package/bundles/antigravity-plugin-extras/skills/schema-migration/SKILL.md

@@ -0,0 +1,275 @@
+---
+name: schema-migration
+context: fork
+description: Migrate an application's data access layer from one database schema to another. Use when tables are renamed, consolidated, split, or columns change — and the app's queries, mappings, and abstraction layer must be updated without data loss. Read-only against the database.
+---
+
+# Schema Migration Skill
+
+Systematically migrate an application from a legacy database schema to a new schema with zero data loss and full functional parity.
+
+## When to Use
+
+- Tables are being renamed, consolidated, or split
+- Columns are renamed, retyped, or moved between tables
+- The DB team has restructured tables and the app must follow
+- Data previously in file storage has been migrated into database tables
+- Query files, column mappings, or config need updating after a schema change
+- User says "migrate to new tables", "schema changed", or "tables restructured"
+
+---
+
+## Constraints (Read Before Starting)
+
+1. **Read-Only** — Never execute DDL or DML (`CREATE`, `ALTER`, `INSERT`, `UPDATE`, `DELETE`) against the database. All analysis is SELECT-only.
+2. **No SQL Aliases** — Proposed queries must use direct `Table.Column` notation. Aliases obscure the mapping and make future changes harder to trace.
+3. **No Assumptions** — Every column mapping must be verified against the actual schema DDL. If a mapping is ambiguous or a source column has no clear target, flag it as a **GAP** rather than guessing.
+4. **Scratchpad Protocol** — If context becomes large, save intermediate findings to `.github/plans/temp_migration_scratchpad.md` to preserve state across reasoning steps.
+
+---
+
+## Phase 1: Schema Forensics
+
+### Objective
+Build a complete picture of old and new schemas. Produce the **Schema Manifest** — the single source of truth for the migration.
+
+### Actions
+
+1. **Collect Legacy Schema DDL**
+   - Extract `CREATE TABLE` statements for all legacy tables (from schema files, INFORMATION_SCHEMA, or DDL scripts)
+   - Note all columns, data types, nullability, defaults, and foreign keys
+   - Document the table-to-variable mapping (e.g., `.env` entries, config files)
+
+2. **Collect Target Schema DDL**
+   - Extract `CREATE TABLE` statements for all new/consolidated tables
+   - Note structural differences: merged tables, renamed columns, new columns, removed columns, type changes
+
+3. **Audit the Abstraction Layer**
+   - Read the app's current mapping files (e.g., `column_mappings.yaml`, `.env`, config modules)
+   - Read the current query files (e.g., `queries.yaml`, `.sql` files)
+   - Identify every table and column the app references
+
+4. **Produce the Mapping Manifest**
+
+```markdown
+## Schema Mapping Manifest
+
+### Table Mapping
+| Legacy Table | Target Table | Notes |
+|-------------|-------------|-------|
+| OldTableA   | NewTable1   | Merged with OldTableB |
+
+### Column Mapping
+| Legacy Table | Legacy Column | Type | Target Table | Target Column | Type | Transform |
+|-------------|--------------|------|-------------|--------------|------|-----------|
+| OldTableA   | user_name    | NVARCHAR(100) | NewTable1 | UserFullName | NVARCHAR(200) | Direct |
+| OldTableA   | status_code  | INT  | NewTable1 | Status | VARCHAR(20) | Map: 1='Active', 2='Inactive' |
+
+### Gaps (Unmapped)
+| Source | Column | Issue |
+|--------|--------|-------|
+| OldTableC | legacy_flag | No equivalent in target schema |
+| File: results/*.json | summary_json | Verify if present in CallTranscript table |
+```
+
+**Output**: Schema Mapping Manifest (table in the migration report)
+
+---
+
+## Phase 2: Query Translation
+
+### Objective
+Rewrite every application query against the new schema. No aliases — direct `Table.Column` references only.
+
+### Actions
+
+1. **Inventory Current Queries**
+   - List every query from query files, repository classes, and inline SQL
+   - Tag each query with its page/feature (e.g., homepage, search, analytics)
+
+2. **Translate Each Query**
+   - Apply the column mapping from Phase 1
+   - Rewrite JOINs to reflect the new table structure
+   - Handle type conversions inline (e.g., `CAST`, `CASE WHEN`) where the new schema uses a different data type
+
+3. **Document Translation**
+
+```markdown
+### Query: [query_name]
+
+**Old Query:**
+```sql
+SELECT col_a, col_b FROM OldTable WHERE col_c = ?
+```
+
+**New Query:**
+```sql
+SELECT NewTable.ColA, NewTable.ColB FROM NewTable WHERE NewTable.ColC = ?
+```
+
+**Changes:**
+- `OldTable` → `NewTable`
+- `col_a` → `ColA` (renamed)
+- No type changes
+```
+
+4. **File Architecture Decision** (if applicable)
+   - If the app uses a monolithic query file, evaluate splitting into per-page/per-feature files
+   - Compare `.yaml` vs `.sql` for query storage:
+
+| Format | Pros | Cons |
+|--------|------|------|
+| `.yaml` | Structured metadata (tags, descriptions), easy Python parsing | No SQL syntax highlighting, IDE support limited |
+| `.sql` | Full IDE support, syntax checking, execution in DB tools | Needs a loader; metadata requires naming conventions or comments |
+
+   - Recommend based on the project's existing patterns and tooling
+
+**Output**: Translated Query Library + File Architecture Recommendation
+
+---
+
+## Phase 3: Parity Testing
+
+### Objective
+Prove that the new queries return the same data as the old queries. Evidence-based — not theoretical.
+
+### Actions
+
+1. **Run Old Queries** — Execute each legacy query and capture:
+   - Row count
+   - Column names and types
+   - Sample rows (5–10 representative rows)
+   - Aggregates where applicable (SUM, COUNT, DISTINCT)
+
+2. **Run New Queries** — Execute each translated query against the new schema and capture the same metrics
+
+3. **Compare Results**
+
+```markdown
+### Parity Report: [query_name]
+
+| Metric | Old Query | New Query | Match |
+|--------|-----------|-----------|-------|
+| Row count | 1,247 | 1,247 | ✅ |
+| Columns | 8 | 8 | ✅ |
+| Sample hash | abc123 | abc123 | ✅ |
+
+**Data Type Check:**
+| Column | Old Type | New Type | Compatible |
+|--------|---------|---------|------------|
+| created_at | DATETIME | DATETIMEOFFSET | ⚠️ App must handle offset |
+```
+
+4. **Flag Discrepancies**
+   - Row count mismatch → possible fan-out from JOINs or missing WHERE filters
+   - Type mismatch → document required CAST/conversion in the app layer
+   - NULL differences → document new nullable columns or default changes
+   - Missing data → flag as GAP with severity
+
+**Output**: Parity Report per query (pass/fail with evidence)
+
+---
+
+## Phase 4: Abstraction Layer Update
+
+### Objective
+Update the application's configuration and mapping layer to reflect the new schema.
+
+### Actions
+
+1. **Update Column Mappings** — Produce the new version of `column_mappings.yaml` (or equivalent) reflecting target table/column names
+
+2. **Update Environment Config** — Produce the new `.env` variable values (table names, schema names)
+
+3. **Update Query Files** — Produce the new query file(s) using the translated queries from Phase 2
+
+4. **Trace Application Code Impact**
+   - Identify every file that reads from the mapping/config layer
+   - List files that will need code changes (e.g., new column names in data models, display logic, filters)
+   - Categorize impact: `no change needed` | `config-only change` | `code change required`
+
+```markdown
+### Impact Analysis
+
+| File | Current Reference | Change Type | Details |
+|------|------------------|-------------|---------|
+| services/data_loader.py | OldTable via .env | Config-only | Update .env mapping |
+| pages/home.py | col_a in display | Code change | Rename to ColA |
+| models/complaint.py | OldTable.status_code (INT) | Code change | Now VARCHAR, update parsing |
+```
+
+**Output**: Updated config files + application impact matrix
+
+---
+
+## Phase 5: Migration Report
+
+### Objective
+Produce a single, comprehensive document suitable for use as an **Implementation Plan**. This document must be complete enough that a developer can execute the migration with zero ambiguity.
+
+### Report Structure
+
+```markdown
+# Schema Migration Report
+
+## Executive Summary
+- Source: [N] legacy tables → Target: [M] consolidated tables
+- Total queries migrated: [count]
+- Parity status: [all pass / N failures]
+- Gaps identified: [count]
+
+## 1. Schema Mapping Manifest
+[From Phase 1]
+
+## 2. Translated Query Library
+[From Phase 2, grouped by page/feature]
+
+## 3. Parity Test Results
+[From Phase 3, pass/fail per query with evidence]
+
+## 4. Abstraction Layer Changes
+[From Phase 4 — updated YAML, .env, config diffs]
+
+## 5. Application Impact Matrix
+[From Phase 4 — file-by-file change list]
+
+## 6. File Architecture Recommendation
+[From Phase 2 — split strategy and format decision]
+
+## 7. Edge-Case Registry
+[Every risk identified during analysis]
+
+| # | Risk | Severity | Mitigation |
+|---|------|----------|------------|
+| 1 | DATETIME → DATETIMEOFFSET | Medium | Add .replace(tzinfo=None) in Python |
+| 2 | Nullable column X now NOT NULL | High | Add default value in query |
+
+## 8. Migration Execution Checklist
+- [ ] Column mappings updated
+- [ ] Environment config updated
+- [ ] Query files updated and tested
+- [ ] Application code changes identified and assigned
+- [ ] All parity tests passing
+- [ ] Edge cases mitigated
+- [ ] Rollback plan documented
+```
+
+### Output Location
+Place the final report at the path specified by the user (default: `.github/plans/backlog/migration_report.md`).
+
+---
+
+## Checklist
+
+- [ ] Legacy schema DDL collected (all tables)
+- [ ] Target schema DDL collected (all tables)
+- [ ] Every legacy column mapped to target (or flagged as GAP)
+- [ ] Every app query translated to new schema
+- [ ] Parity tests executed (old vs new query results compared)
+- [ ] No row count mismatches unexplained
+- [ ] Data type changes documented with app-level mitigation
+- [ ] Abstraction layer (mappings, config) updated
+- [ ] Application code impact traced (file-by-file)
+- [ ] Edge-case registry complete (no known risks undocumented)
+- [ ] File architecture decision documented (split strategy, format)
+- [ ] Report placed at specified output path

package/bundles/antigravity-plugin-extras/skills/setup-project-ai/SKILL.md

@@ -0,0 +1,120 @@
+---
+name: setup-project-ai
+description: "Install or refresh the agent-agnostic development harness in a project — an AGENTS.md-canonical rules hierarchy (root + per-folder) with a CLAUDE.md @import shim beside each, lean subagents, slash commands, settings.json, and the frontend/python test env. USE THIS SKILL when setting up AI resources on a new or existing project, bootstrapping the rules files (AGENTS.md/CLAUDE.md), adding subagents/commands, or when the user says 'set up the harness', 'setup-project-ai', 'generate CLAUDE.md', 'onboard this project to Claude Code / Codex / agy', or runs /setup-project-ai. Combines a deterministic generator (scripts/setup_project_ai.py) for the must-not-vary mechanics with an AI enrichment step that curates project-specific AGENTS.md content."
+---
+
+# setup-project-ai
+
+Produce a working, TDD-first, context-rot-resistant dev harness for any project. Two layers:
+**deterministic** (a Python generator — mechanics that must not vary) + **generative** (you — curating
+project-specific AGENTS.md content and verifying). Proven in the Phase 0 spike.
+
+## When to use
+- New project (e.g. freshly bootstrapped from a template) needs its Claude Code harness.
+- Existing project moving to Claude Code / Codex.
+- Refreshing the harness after the canonical templates improved (`--force`).
+
+## What it produces
+```
+AGENTS.md                 canonical root rules (identity + harness loop + laws + pointers) — every agent reads it
+CLAUDE.md                 root @AGENTS.md import shim + a small Claude-native block (subagents/skills/commands)
+src/AGENTS.md             backend conventions   (when a Python/FastAPI backend is detected)  + src/CLAUDE.md shim
+frontend/AGENTS.md        frontend conventions  (when React/Vite is detected)                + frontend/CLAUDE.md shim
+tests/AGENTS.md           test/TDD conventions  (when pytest/tests are detected)             + tests/CLAUDE.md shim
+.claude/agents/*          lean subagents: tester, code-reviewer, preflight
+.claude/commands/*        feature-plan, tdd, prd, handoff, setup-project-ai
+.claude/settings.json     stack-tuned permissions (pipeline MCP off by default)
+```
+Each `CLAUDE.md` is a thin `@AGENTS.md` import shim — Claude Code inlines the sibling AGENTS.md at load,
+so the rules are single-sourced. Codex and Antigravity (`agy`) read the `AGENTS.md` files directly.
+
+## Procedure
+
+### Step 1 — Run the deterministic generator
+**Resolve `<harness-root>` first** — the generator ships in two places and finds its own templates:
+- **Installed as the caddis plugin (the common case):** `<harness-root>` = `${CLAUDE_PLUGIN_ROOT}`,
+  so the script is `${CLAUDE_PLUGIN_ROOT}/scripts/setup_project_ai.py`. Templates (`claude-md/`,
+  `settings.template.json`, `stack-map.json`) sit at the plugin root and are auto-located.
+- **harness dev checkout:** `<harness-root>` = the caddis repo root; the script is
+  `scripts/setup_project_ai.py` with templates in the sibling `claude-harness/`.
+
+Run a dry-run first to see stack detection and unresolved placeholders:
+
+```
+python <harness-root>/scripts/setup_project_ai.py <target-project-dir> \
+    --name "<Display Name>" --desc "<one-line description>" --dry-run
+```
+
+Read the output. It reports the detected stack, the AGENTS.md rules hierarchy (+ CLAUDE.md shims) it will write, the subagents/
+commands it will deploy, and — critically — **unresolved `{{PLACEHOLDER}}` tokens** grouped by file
+(Phase 0 friction #1: these break runtime if left, e.g. ports in `settings.py`/`vite.config.ts`).
+
+### Step 2 — Run for real (and substitute only for fresh template copies)
+**Existing project (the common case):** placeholder substitution is OFF by default — the generator
+reports `{{TOKENS}}` it finds (informational; usually just example text in docs) but **never rewrites
+repo files**. Just run without `--dry-run`:
+```
+python <harness-root>/scripts/setup_project_ai.py <target-project-dir> \
+    --name "<Display Name>" --desc "<one-line description>"
+```
+
+**Fresh-from-template project only:** the copy has real `{{API_PORT_DEV}}` etc. that break runtime
+(Phase 0 friction #1). Resolve them from the platform port registry / `project-config.md` and pass
+`--substitute` plus each `--set`:
+```
+python <harness-root>/scripts/setup_project_ai.py <target-project-dir> \
+    --name "<Display Name>" --desc "<one-line description>" --substitute \
+    --set PROJECT_NAME=<slug> --set API_PORT_DEV=<n> --set FE_PORT_DEV=<n> --set PROJECT_SHORT=<short> ...
+```
+> ⚠ **Never pass `--substitute` on an existing repo** — it would rewrite docs/code containing `{{...}}`
+> as literal examples (dogfood learning from a live incident on an existing project). Substitution is a
+> template-bootstrap step.
+
+Add `--force` only when intentionally overwriting an existing CLAUDE.md/AGENTS.md/harness file.
+Confirm zero runtime placeholders remain: the runtime files (`settings.py`, `vite.config.ts`,
+`main.py`) must compile / parse. Quick check: `python -m py_compile src/config/settings.py`.
+
+### Step 3 — Enrich the AGENTS.md hierarchy (the AI step — this is the value-add)
+The generator writes **generic** fragments and pre-extracts real facts into
+`.caddis/PROJECT-FACTS.md` (run/test/build commands, env-var names, CI/deploy workflows, entry
+points). **Start there** — fold each fact into the *right* `AGENTS.md` (root vs `src/` vs
+`frontend/` vs `tests/`) — the **canonical** rules file, never the `CLAUDE.md` shim — then delete
+`PROJECT-FACTS.md`. Then read `STACK.md` (if present), `project-config.md`, and skim the actual code to
+add the project-specific truth the generic fragments can't know:
+- Real shared-library import paths, the actual data-access layer, domain terms/glossary.
+- The project's **actual** run/test/build commands (from `package.json`/`pyproject.toml`/scripts).
+- Concrete patterns to mirror (name a real exemplar file: "follow `src/services/<x>_store.py`").
+- Anything load-bearing and non-obvious (auth model, env files, DB topology).
+Keep the root **lean** — push depth into the folder `AGENTS.md` files and into skills. Do not duplicate
+the generic laws/harness (already in the root AGENTS.md). The goal is the best-of-best AGENTS.md per
+repo. Leave every `CLAUDE.md` as the generated shim — enriching it would fork the rules for Claude only.
+
+### Step 4 — Ensure the test environment (Phase 0 frictions #3/#4)
+- **Python:** if `.venv` missing, create it and install dev deps:
+  `python -m venv .venv` then `.venv/Scripts/pip install -e .[dev]` (Windows) / `.venv/bin/pip ...`.
+- **Frontend:** if the generator scaffolded the Vitest/jsdom harness, run `npm install` in the frontend
+  so `jsdom` + `@testing-library/jest-dom` actually install, and confirm `vite.config` has a `test`
+  block (the generator emits the exact snippet to add if missing).
+
+### Step 5 — Smoke-test the harness (verify, don't assume — Law 4)
+Prove the harness is live before declaring done:
+- Run the existing suite (`pytest -q`, `npm test`) — or write one trivial passing test — and show output.
+- Optionally dispatch the `preflight` subagent on any existing plan to confirm subagent dispatch works.
+
+### Step 6 — Report + relay
+Summarize what was created, the detected stack, any deferred placeholders, and the smoke-test result.
+If this is a multi-session effort, run `/handoff` to write `relay.md`.
+
+## Notes
+- **Idempotent.** Re-running preserves existing AGENTS.md/CLAUDE.md/settings unless `--force`; settings
+  allow-lists are always merged (union). Safe to re-run.
+- **Pipeline is off by default.** The lightweight loop (plan file + relay) is the default. Enable the
+  optional MCP pipeline power-mode only for large multi-week features (add `mcpServers.junai` +
+  `.github/pipeline-state.json`).
+- **Agent-agnostic.** `AGENTS.md` is the canonical rules file; `CLAUDE.md` is a thin `@AGENTS.md` import
+  shim (Claude Code inlines it). subagents/commands/skills are plain markdown Codex/agy can also read.
+  The same harness serves Claude Code, Codex CLI, and Antigravity (`agy`).
+- **Packaging:** the generator ships **inside the caddis plugin** (`scripts/setup_project_ai.py`
+  with the templates at the plugin root), so `/setup-project-ai` runs from a plain plugin install with
+  no harness checkout. The same script also runs from the harness repo for harness development.
+  Resolve its path via `<harness-root>` as described in Step 1.