vinta-ai-workflows 0.2.0 → 0.4.0

package/skills/vinta-derive-skills/resources/foundation-skills/prepare-worktree/SKILL.md

@@ -109,6 +109,8 @@ Default: **copy** for pnpm + npm + cargo + go; **reinstall** for poetry + venv +
 
 `env_change = true` → `cp <main>/.env <worktree>/.env`. Then mutate per the **Database fork** step (DB URL) + the **Docker / compose isolation** step (compose project name).
 
+**Compose forces a copy.** If the project uses compose, the [4a](#4a--neutralize-compose-isolation-leaks) sub-step appends `COMPOSE_FILE=…` to the worktree's `.env` — a mutation. So a compose project ALWAYS needs a copied `.env` (or a copied `.envrc`) even when `env_change = false`; a symlinked `.env` would leak the `COMPOSE_FILE` line into main. Treat "project uses compose" as an independent reason to copy `.env`.
+
 `.env.example` is **always** symlinked — it's committed; the worktree should keep tracking it.
 
 ### 2c — Skip rules
@@ -129,6 +131,11 @@ Whatever the app reads + writes during local dev. Detect the connection string s
 2. Project config files: `config/database.yml` (Rails), `database.ini`, `prisma/schema.prisma`'s `datasource`, `knexfile.js`, Django `settings.py`'s `DATABASES`.
 3. Docker compose service env: `services.db.environment.POSTGRES_DB`.
 
+**First, determine HOW the DB is delivered — this changes what "share" even means.** Two deliveries:
+
+- **External server** — the DB runs on a server the worktree only *connects* to (a system Postgres, an RDS instance, a compose service the worktree does NOT boot its own copy of). Here "share" = point the worktree's connection string at the one already-running server. Fork = clone into a new DB *on that same server*.
+- **Per-worktree compose stack** — the DB is delivered *through the worktree's own `docker compose up`* (a `db` service the worktree boots). Here the worktree runs its **own server process**. "Share" the way the connection-string sense means is impossible without corruption: booting a second server against the same data volume is two postmasters on one PGDATA. **For compose-delivered DBs the data volumes are ALWAYS forked, independent of `schema_change`** — see the [Neutralize compose isolation leaks](#4a--neutralize-compose-isolation-leaks) sub-step, which does the volume fork mechanically. `schema_change = false` may still let you skip *cloning main's data* into the forked volume (boot empty + migrate), but it never authorizes a shared data volume.
+
 For each detected DB, ask the user (`AskUserQuestion`) once:
 
 - **Fork the DB** — recommended when `schema_change = true` or when the plan's phases run destructive migrations. Strategy depends on the engine:
@@ -137,7 +144,7 @@ For each detected DB, ask the user (`AskUserQuestion`) once:
   - **SQLite (file-based)** — `cp <main>/db.sqlite3 <worktree>/db.sqlite3`. Symlink would defeat the point.
   - **Mongo / Redis / Elasticsearch** — engine-specific clone, OR per-worktree DB name / key prefix (`REDIS_URL=redis://localhost:6379/<index>` with a free index; `MONGODB_URI=mongodb://localhost:27017/<db>_wt_<name>`).
 
-- **Share the DB (don't fork)** — only safe when `schema_change = false` AND there's no risk of conflicting writes. Symlink the SQLite file or point the worktree's env at the same URL. Warn explicitly: "DB shared with main checkout — destructive ops here will be visible in main."
+- **Share the DB (don't fork)** — means share a *connection to a single already-running server*, nothing more. Only safe when the DB is an **external server** (NOT compose-delivered — see the delivery check above) AND `schema_change = false` AND there's no risk of conflicting writes. Symlink the SQLite file or point the worktree's env at the same URL. Warn explicitly: "DB shared with main checkout — destructive ops here will be visible in main." **Never read this bullet as permission to point a second per-worktree compose server at a shared data volume** — that is not sharing a connection, it is running two servers on one data directory, and it corrupts the store regardless of `schema_change`.
 
 - **Stub the DB** — point the worktree at a fresh empty DB (`createdb <main_db>_wt_<name>` + run all migrations + seed if a seed exists). Best when the feature needs schema parity but no production data.
 
@@ -147,10 +154,12 @@ Record the chosen strategy + the forked DB name in the worktree summary (written
 
 A different beast — tests on the same engine but a different DB name (`<main_db>_test`, `test_<repo>`). Parallel worktrees running tests against the same `<main_db>_test` will overwrite each other's fixtures and produce flaky failures.
 
-- **`pytest-django` / `pytest`** — set `--reuse-db` per worktree via a per-worktree `DJANGO_SETTINGS_MODULE` env, OR override `DATABASES['default']['NAME']` to `<main_db>_test_wt_<name>` in `conftest.py` when the env var `WORKTREE_NAME` is set. Drop a `conftest_worktree.py` patch in the worktree (don't edit `conftest.py` in tracked code — too easy to commit by accident).
-- **`vitest` / `jest`** — set `TEST_DATABASE_URL` per worktree; ensure the test setup respects it.
-- **Rails** — `DATABASE_URL` for the `test` env, `<main_db>_test_wt_<name>`.
-- **Docker-compose-based test DB** — see the **Docker / compose isolation** step below; per-worktree compose project name fixes it.
+**The principle (stack-agnostic):** give the worktree its own test DB name (`<main_db>_test_wt_<name>`), inject it through whatever channel the test runner already reads (an env var or a per-worktree config override), and **never edit tracked test config in place** — a worktree-local override or an env var is safe; a change to a committed config file is one `git add -A` away from leaking into main. How each runner exposes that channel:
+
+- **Env-var runners (`vitest` / `jest`, `go test`, `cargo test`, most CI-shaped suites)** — set `TEST_DATABASE_URL` (or the suite's equivalent) per worktree and confirm the test setup reads it. Simplest case; no file edits at all.
+- **Config-file runners (`pytest-django` / `pytest`)** — set `--reuse-db` per worktree via a per-worktree `DJANGO_SETTINGS_MODULE`, OR override `DATABASES['default']['NAME']` to `<main_db>_test_wt_<name>` guarded on a `WORKTREE_NAME` env var. Put the override in a `conftest_worktree.py` in the worktree — don't touch the tracked `conftest.py`.
+- **Convention-based runners (Rails)** — point the `test` env's `DATABASE_URL` at `<main_db>_test_wt_<name>`.
+- **Docker-compose-based test DB (any stack)** — see the **Docker / compose isolation** step below; a per-worktree compose project name fixes it.
 
 If `test_infra_change = true` → fork the test DB unconditionally. If `false` and the user says "share is fine" → still flag the race risk; offer a one-line fix to switch later.
 
@@ -166,10 +175,15 @@ If the project has a `docker-compose.yml` / `compose.yaml` / `docker-compose.ove
 
 1. Set `COMPOSE_PROJECT_NAME=<repo>_<worktree-name>` in the worktree's `.env` (or in a `.envrc` if the project uses direnv). Then `docker compose up` inside the worktree spins a fresh container set.
 
-2. **Shared network** — if the worktree's app needs to reach services running in the *main* checkout's compose stack (a queue, a cache, a search index that's expensive to spin twice), put them on an external network:
+   **`COMPOSE_PROJECT_NAME` is necessary but NOT sufficient.** It namespaces containers and networks, and auto-named volumes — but it does **not** isolate two very common compose patterns: volumes declared `external: true` and/or pinned to a fixed top-level `name:`, and services with fixed host port bindings (`ports: - "5432:5432"`). A shared *database* volume is the dangerous one: two worktree stacks (or a worktree stack + main's stack) mount the same physical volume, so two server processes run against one data directory → corruption. The next sub-step closes both leaks mechanically.
+
+2. **Neutralize compose isolation leaks** — see [4a](#4a--neutralize-compose-isolation-leaks) below. Run this whenever the project has compose, before ever booting the worktree's stack.
+
+3. **Shared network** — if the worktree's app needs to reach services running in the *main* checkout's compose stack (a queue, a cache, a search index that's expensive to spin twice), put them on an external network:
 
    ```yaml
-   # docker-compose.override.yml in the worktree (or in main; pick one place)
+   # Add to the SAME out-of-tree override the 4a sub-step generates — NOT a
+   # tracked docker-compose.override.yml at the worktree root (see 4a for why).
    networks:
      shared:
        external: true
@@ -182,9 +196,44 @@ If the project has a `docker-compose.yml` / `compose.yaml` / `docker-compose.ove
 
    Or use `network_mode: host` if the project uses that pattern. Decide once per project and record in `.vinta-ai-workflows.yaml` → `run_options.prepare-worktree.compose_network: per-worktree | shared-external | host`.
 
-3. **Linters / formatters / test runners that run inside docker** (a Dockerfile-based `pnpm test`, a `lint` target that mounts source into a sidecar container) — these MUST run with the worktree's `COMPOSE_PROJECT_NAME`. Otherwise concurrent runs hit the same container and one wins. The standard fix: ensure the project's lint/test scripts read `COMPOSE_PROJECT_NAME` from env instead of hard-coding it.
+4. **Linters / formatters / test runners that run inside docker** (a Dockerfile-based `pnpm test`, a `lint` target that mounts source into a sidecar container) — these MUST run with the worktree's `COMPOSE_PROJECT_NAME`. Otherwise concurrent runs hit the same container and one wins. The standard fix: ensure the project's lint/test scripts read `COMPOSE_PROJECT_NAME` from env instead of hard-coding it.
+
+5. **Per-worktree volumes** — for a volume the project leaves un-named, compose auto-namespaces it by project name and it's already isolated. But projects routinely pin data volumes with `external: true` or a fixed `name:` (for good reasons of their own — you must NOT edit their committed compose to "fix" that). Those are the leaks the [4a](#4a--neutralize-compose-isolation-leaks) sub-step re-pins in a generated override. Never touch the tracked compose file to change a volume's isolation; the override is the only lever.
+
+### 4a — Neutralize compose isolation leaks
+
+Runs whenever the project has any compose file, before the worktree's stack is ever booted. Fully project-agnostic — every decision is driven by `docker compose config --format json`, no project / service / volume name is hardcoded.
+
+**1. Generate the override.** Run the bundled [scripts/gen-compose-worktree-override.sh](scripts/gen-compose-worktree-override.sh):
+
+```bash
+override=$(scripts/gen-compose-worktree-override.sh \
+  --main     <main-checkout-root> \
+  --worktree <worktree-path> \
+  --name     <worktree-name> \
+  --out      <summary_dir>/<name>.docker-compose.override.yml \
+  --share-volume <volkey>...)   # 0+ volumes safe to keep shared (see below)
+```
+
+The script renders the resolved config from the main checkout and, for the worktree:
+- re-pins every volume that is `external: true` OR carries a fixed top-level `name:` to a **non-external, worktree-namespaced** volume (`<project>_wt_<name>_<volkey>`), so the worktree's server writes to its OWN physical volume;
+- strips host port publishing from every service that publishes a fixed port (via compose's `!override []` — a plain `ports: []` **merges/appends** in Compose v2+ and would NOT strip it; verified on Docker Compose v5.1.1);
+- leaves alone any volume already namespaced by project name (auto-named), and any volume the caller passes via `--share-volume` (a read-only dep/venv cache with no dep churn is safe to keep shared — the config's `shared_volumes` list feeds this arg).
+
+The script exits non-zero with a clear message if `docker compose` is unavailable or the config doesn't parse; surface that and stop (don't boot a stack you couldn't prove is isolated).
+
+**2. Write it OUT OF TREE and wire it in via the copied `.env`.** The override goes to `<summary_dir>/<name>.docker-compose.override.yml` (under the already-ignored `.vinta-ai-workflows/` umbrella) — **NOT** a `docker-compose.override.yml` at the worktree root. Rationale: a root `docker-compose.override.yml` is auto-loaded by compose, but it is frequently a **tracked** file (it was in the project that hit the 2026-07-17 incident); writing the generated override there pollutes the worktree diff and risks `git add -A` leaking it into main. Instead, wire it in by appending to the worktree's **copied** `.env`:
+
+```bash
+# The generator prints exactly this line (base:override) on stderr — use it verbatim.
+echo "COMPOSE_FILE=<base-compose-file>:<ABSOLUTE-override-path>" >> <worktree>/.env
+```
+
+`COMPOSE_FILE` makes compose load base + override for every `docker compose` call in the worktree without touching any tracked file. The **override path must be absolute** — it lives under `<summary_dir>` (outside the worktree), so a bare basename wouldn't resolve from the worktree's compose project dir; the generator prints the absolute path for you (and on stdout, for `$(...)` capture). The base file stays relative — the worktree carries its own tracked copy. **This requires `.env` to be COPIED, not symlinked** — appending to a symlinked `.env` leaks the line into main. If the **Env files** step decided `env_change = false` (symlink), flip `.env` to a copy here (compose isolation is itself an env change) and note it in the summary. Use the exact base compose filename the generator reports (`compose.yaml` vs `docker-compose.yml`).
+
+**3. Record the decision.** Write the per-volume fork decisions and the override path into the summary YAML's `state.compose` (see the **Write the summary file** step). The override's own `volumes:` block is the teardown manifest — each pinned `name:` is a `docker volume rm` target.
 
-4. **Per-worktree volumes** — let compose auto-create volumes namespaced by project name. Don't manually `external: true` data volumes (defeats isolation).
+**Relation to `test_db_strategy`.** The `test_db_strategy: fork-on-schema-change` config governs the **test-DB axis only** (Step 3b — which test DB *name* the runner targets). It says nothing about compose data volumes and must **never** be read as permission to share the compose stack's volumes. Volume forking here is unconditional for compose-delivered DBs, independent of `schema_change` and independent of `test_db_strategy`.
 
 ## Step 5 — Other shared infra to fork
 
@@ -303,6 +352,14 @@ state:
   compose:
     project_name: <repo>_<worktree-name>
     network_strategy: per-worktree | shared-external | host
+    override_path: <summary_dir>/<name>.docker-compose.override.yml   # null when no compose
+    base_compose_file: docker-compose.yml    # the base file COMPOSE_FILE prepends
+    forked_volumes:                          # [] when nothing leaked; teardown removes each name
+      - volume_key: pgdata                   # the compose key
+        reason: external | fixed-name        # why it leaked past COMPOSE_PROJECT_NAME
+        forked_name: <repo>_wt_<name>_pgdata # the worktree-namespaced physical volume
+    shared_volumes: [virtualenv]             # leaky volumes deliberately kept shared (--share-volume)
+    ports_stripped_from: [db, cache]         # services whose fixed host ports the override removed
   other:
     redis_db_index: <int>
     s3_prefix: wt-<name>/
@@ -346,7 +403,12 @@ guard is active; the orchestrator's post-run stray-write check is the backstop.
 When the plan is merged / abandoned:
   git worktree remove <path>
   <drop-db command>          # if fork strategy was `fork`
-  docker compose -p <project_name> down -v   # if compose was forked
+  # Compose (if forked). Do NOT use `down -v` — it can also nuke SHARED volumes
+  # (a fixed-name dep/venv cache). Bring the stack down, then remove only the
+  # worktree-namespaced volumes the override created:
+  docker compose -p <project_name> down
+  docker volume rm <each state.compose.forked_volumes[].forked_name>
+  rm <override_path>         # the generated out-of-tree compose override
 
 The summary file at `.vinta-ai-workflows/worktrees/<name>.yaml` records every
 decision for mechanical teardown.
@@ -371,7 +433,10 @@ A sibling skill / explicit command — not auto-run by this skill. Steps:
 1. Confirm no uncommitted changes in the worktree (`git -C <path> status`).
 2. `git worktree remove <path>` (or `--force` after explicit confirmation if the user is fine losing work).
 3. Drop the forked DB (`dropdb <forked_name>`, `DROP DATABASE`, `rm <sqlite-file>`, …) — read the strategy from the summary YAML.
-4. `docker compose -p <project_name> down -v` for forked compose.
+4. Compose teardown (read `state.compose` from the summary YAML):
+   - `docker compose -p <project_name> down` — **not `down -v`**; a bare `down -v` also removes shared volumes (a fixed-name dep/venv cache in `state.compose.shared_volumes`), which belong to main.
+   - `docker volume rm <forked_name>` for each entry in `state.compose.forked_volumes` — these are the worktree-only volumes the generated override created; removing them is safe and leaves main's / shared volumes untouched.
+   - `rm <state.compose.override_path>` — delete the generated out-of-tree override.
 5. Remove `.vinta-ai-workflows/worktrees/<name>.yaml`.
 
 Every step gated on user confirmation when the worktree has un-pushed branches.
@@ -380,6 +445,7 @@ Every step gated on user confirmation when the worktree has un-pushed branches.
 
 - **Symlink for reads, copy for writes, fork for state.** This is the only mental model that scales. Default to fork when unsure — disk is cheap, corrupted main-checkout DBs are not.
 - **Never share a writable DB across worktrees by default.** The race conditions are subtle and the failure mode is silent data corruption.
+- **`COMPOSE_PROJECT_NAME` does not isolate `external:`/fixed-`name:` volumes or fixed host ports — always run the [4a](#4a--neutralize-compose-isolation-leaks) generator.** For a compose-delivered DB the data volumes are forked *unconditionally* (independent of `schema_change` and `test_db_strategy`): the worktree runs its own server, and two servers on one volume is corruption. "Share the DB" only ever means share a connection to a single already-running server, never a second server on shared storage. The fix is a generated, out-of-tree override + a `COMPOSE_FILE` line in the copied `.env` — never an edit to any tracked compose file. Verify the volume-differs invariant before declaring the worktree runnable.
 - **Every fork decision lands in `.vinta-ai-workflows/worktrees/<name>.yaml`.** Teardown reads it; humans grep it; agents resuming a stalled plan read it. No decision lives only in conversation memory.
 - **Worktree root governed by runtime conventions.** claude-code uses `.claude/worktrees/`; other harnesses use sibling dirs. Don't fight the harness — match it.
 - **Don't mutate the main checkout from this skill.** Every write goes to the worktree or to `.vinta-ai-workflows/`. Forking a Postgres DB is the one exception (the new DB lives in the same server) — document it loudly in the summary.
@@ -395,6 +461,9 @@ Every step gated on user confirmation when the worktree has un-pushed branches.
 - **Symlinking `node_modules` for a `pnpm add` phase.** The new package writes back through the symlink into the main checkout's store. Detect dep churn in the **Plan inspection** step and copy/reinstall instead.
 - **Forking the dev DB but forgetting the test DB.** Tests still hit the shared test DB and stomp on parallel worktrees' fixtures. Both axes need their own decision in the **Database fork** step.
 - **Forgetting `COMPOSE_PROJECT_NAME`.** Containers from worktree-A overwrite worktree-B's containers; volumes get nuked. Set it in `.env` so every `docker compose` call inherits.
+- **Trusting `COMPOSE_PROJECT_NAME` to isolate `external:`/fixed-`name:` volumes and fixed host ports — it does NOT.** These defeat the project-name namespace: every worktree's compose stack mounts the SAME external/named volume and binds the SAME host port. For a DB volume that means two server processes on one data directory — the 2026-07-17 corruption incident (a worktree's `docker compose down` deleted the shared `postmaster.pid` and the main checkout's Postgres self-terminated). The fix is the [4a](#4a--neutralize-compose-isolation-leaks) generated out-of-tree override, never an edit to the tracked compose file. Verify the invariant (Verification step 6), don't assume it.
+- **Confusing "share the DB" (a connection to one running server) with "share the data volume" (a second server on shared storage).** The first can be safe when `schema_change = false`; the second is always corruption. `schema_change = false` and `test_db_strategy` never authorize a shared compose data volume — see [3a](#3a--dev--app-database).
+- **Writing the compose override to the worktree-root `docker-compose.override.yml`.** It's auto-loaded (convenient) but often a *tracked* file — the generated override then shows up in the diff and one `git add -A` leaks it into main. Write it under `<summary_dir>/` and wire it in via `COMPOSE_FILE` in the COPIED `.env` ([4a](#4a--neutralize-compose-isolation-leaks)).
 - **Sharing a Redis DB without per-worktree prefix.** Tests writing `user:123` collide across worktrees. Pick an index OR a key prefix.
 - **Copying `node_modules` for yarn PnP / absolute-path setups.** The copy carries baked-in paths from the main checkout. Reinstall instead — pnpm's relative-symlink store is the safe-to-copy exception.
 - **Forgetting to `--allow` the `.vinta-ai-workflows` dir under sandbox.** It's shared/symlinked but the run *writes* it (summary YAMLs, prs-context). If it's not an `--allow` exception, those writes hit the read-only main subtree and fail mid-run. Pass it alongside the worktree path.
@@ -413,4 +482,7 @@ After the **Write the summary file** step writes the summary:
 3. `git -C <worktree-path> status` is clean (no accidental file additions from the prep step).
 4. The summary YAML parses (`python3 -c "import yaml; yaml.safe_load(open('.vinta-ai-workflows/worktrees/<name>.yaml'))"`).
 5. `WORKTREE.md` exists at the worktree root with accurate fork / share annotations.
-6. Optional smoke test: run a single new-test command in the worktree (e.g. `pytest -x tests/health.py`) — confirms env vars resolved, DB reachable, deps importable.
+6. **Compose volume-isolation invariant (when the DB is compose-delivered).** This is the checked form of the footgun that caused the 2026-07-17 incident — do not skip it when the project boots its DB via compose. From the worktree:
+   - Resolve the worktree's DB data volume: `docker compose config --format json | <read the db service's data volume `source`, then that volume's `.name`>`. Assert it **differs** from the main checkout's resolved DB volume name (run the same read in `<main-checkout-root>`). If they're equal, the override didn't take — **fail provisioning loudly**; do not proceed.
+   - Boot only the DB service in the worktree (`docker compose up -d <db-service>`), then confirm the **main checkout's** DB container is still healthy/running afterward (`docker inspect`/`pg_isready` against main's DB). Bring the worktree DB back down. A main-DB disturbance here means isolation failed — fail provisioning.
+7. Optional smoke test: run a single new-test command in the worktree (e.g. `pytest -x tests/health.py`) — confirms env vars resolved, DB reachable, deps importable.

package/skills/vinta-derive-skills/resources/foundation-skills/prepare-worktree/scripts/gen-compose-worktree-override.sh

@@ -0,0 +1,211 @@
+#!/usr/bin/env bash
+# gen-compose-worktree-override.sh — generate an out-of-tree docker-compose
+# override that neutralizes the isolation leaks COMPOSE_PROJECT_NAME does NOT
+# cover, so a per-worktree compose stack cannot collide with the main checkout's.
+#
+# Why this exists: COMPOSE_PROJECT_NAME namespaces containers and networks, but
+# NOT two very common compose patterns —
+#
+#   1. Volumes declared `external: true` and/or pinned to a fixed top-level
+#      `name:`. These are addressed by name, so EVERY worktree's compose stack
+#      mounts the SAME physical volume. For a database volume that means two
+#      postmasters on one PGDATA → data corruption (the 2026-07-17 incident).
+#   2. Fixed host port bindings (`ports: - "5432:5432"`). These publish to a
+#      fixed host port that collides across compose projects regardless of name.
+#
+# The fix is a GENERATED, out-of-tree override (never a mutation of the project's
+# committed compose files) that, for the worktree:
+#   - re-pins every leaky volume to a NON-external, worktree-namespaced volume
+#     (`<project>_wt_<name>_<volkey>`), EXCEPT volumes the caller marks shareable
+#     (a read-only dep/venv cache with no dep churn is safe to keep shared);
+#   - strips host port publishing from every service that has it, using compose's
+#     `!override []` tag (a plain `ports: []` MERGES/appends in Compose v2+ and
+#     would NOT strip the port — verified on Docker Compose v5.1.1).
+#
+# Detection is 100% generic — no project / service / volume name is hardcoded.
+# Everything comes from `docker compose config --format json` run in the main
+# checkout. A volume leaks when `.external == true` OR its resolved `.name`
+# differs from the auto-namespaced form `<project>_<volkey>` (i.e. it carries a
+# fixed top-level `name:`). Setting `external: false` in the override is
+# mandatory — without it the base `external: true` merges through and the volume
+# stays externally-pinned (verified).
+#
+# Usage:
+#   gen-compose-worktree-override.sh \
+#     --main     <main-checkout-root> \
+#     --worktree <worktree-path> \
+#     --name     <worktree-name> \
+#     --out      <override-output-path> \
+#     [--share-volume <volkey>]...   # volumes safe to keep shared (repeatable)
+#
+# Output:
+#   stdout — the override path (only, on success) so it is safe to capture with
+#            $(...). stderr carries a human-readable decision report + the
+#            suggested `COMPOSE_FILE=<base>:<override>` wiring line.
+#   The generated override's `volumes:` block lists the forked volumes by their
+#   pinned name — that IS the teardown manifest (`docker volume rm <name>`).
+#
+# Exit: 0 on success; 2 on argument / setup errors (no `docker compose`, config
+#       does not parse, no base compose file found).
+
+set -u
+
+die() { echo "gen-compose-worktree-override: $*" >&2; exit 2; }
+
+MAIN=""; WORKTREE=""; NAME=""; OUT=""
+SHARE=()
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --main)         shift; [ $# -gt 0 ] || die "--main needs a path"; MAIN="$1"; shift ;;
+    --worktree)     shift; [ $# -gt 0 ] || die "--worktree needs a path"; WORKTREE="$1"; shift ;;
+    --name)         shift; [ $# -gt 0 ] || die "--name needs a value"; NAME="$1"; shift ;;
+    --out)          shift; [ $# -gt 0 ] || die "--out needs a path"; OUT="$1"; shift ;;
+    --share-volume) shift; [ $# -gt 0 ] || die "--share-volume needs a volume key"; SHARE+=("$1"); shift ;;
+    *)              die "unknown arg: $1" ;;
+  esac
+done
+
+[ -n "$MAIN" ]     || die "--main <main-checkout-root> required"
+[ -n "$WORKTREE" ] || die "--worktree <path> required"
+[ -n "$NAME" ]     || die "--name <worktree-name> required"
+[ -n "$OUT" ]      || die "--out <override-path> required"
+
+command -v docker >/dev/null 2>&1 || die "docker not found on PATH — cannot generate compose override"
+docker compose version >/dev/null 2>&1 || die "'docker compose' unavailable — is the Compose v2 plugin installed?"
+
+# Canonicalize the main checkout (must exist).
+[ -d "$MAIN" ] || die "main checkout path is not a directory: $MAIN"
+MAIN=$(cd "$MAIN" && pwd -P)
+
+# Locate the base compose file compose will auto-load (standard search order).
+BASE=""
+for f in compose.yaml compose.yml docker-compose.yaml docker-compose.yml; do
+  if [ -f "$MAIN/$f" ]; then BASE="$f"; break; fi
+done
+[ -n "$BASE" ] || die "no compose file (compose.yaml / docker-compose.yml) found in $MAIN"
+
+# Render the resolved config from the MAIN checkout. Do not force
+# COMPOSE_PROJECT_NAME: let it default exactly as the project normally resolves
+# it, so the auto-namespace comparison below reflects reality.
+CONFIG_JSON=$(cd "$MAIN" && docker compose config --format json 2>/dev/null) \
+  || die "'docker compose config' failed in $MAIN — the compose config does not parse"
+
+mkdir -p "$(dirname "$OUT")" || die "cannot create output dir for $OUT"
+
+CONFIG_JSON="$CONFIG_JSON" OUT="$OUT" NAME="$NAME" BASE="$BASE" \
+python3 - "${SHARE[@]:-}" <<'PY' || die "failed to generate override"
+import json, os, sys
+
+cfg = json.loads(os.environ["CONFIG_JSON"])
+out = os.environ["OUT"]
+wt_name = os.environ["NAME"]
+base = os.environ["BASE"]
+share = {s for s in sys.argv[1:] if s}
+
+# The override lives OUT of the worktree (under <summary_dir>), so the COMPOSE_FILE
+# entry must reference it by ABSOLUTE path — a basename would only resolve if the
+# file sat in the compose project dir. The base file is worktree-local (the
+# worktree carries its own tracked copy), so it stays relative.
+out_abs = os.path.abspath(out)
+
+project = cfg.get("name") or "compose"
+volumes = cfg.get("volumes") or {}
+services = cfg.get("services") or {}
+
+# ---- Detect leaky volumes -------------------------------------------------
+# A volume leaks past COMPOSE_PROJECT_NAME when it is external, or carries a
+# fixed top-level name (resolved name != the auto-namespaced <project>_<key>).
+forked = []          # (volkey, reason, pinned_name)
+shared_leaky = []    # (volkey, reason) — leaky but caller marked shareable
+for key, vol in volumes.items():
+    vol = vol or {}
+    external = bool(vol.get("external"))
+    resolved = vol.get("name", "")
+    auto = f"{project}_{key}"
+    if external:
+        reason = "external:true"
+    elif resolved and resolved != auto:
+        reason = f"fixed name '{resolved}'"
+    else:
+        continue  # already namespaced by project name — safe
+    if key in share:
+        shared_leaky.append((key, reason))
+        continue
+    pinned = f"{project}_wt_{wt_name}_{key}"
+    forked.append((key, reason, pinned))
+
+# ---- Detect services publishing fixed host ports --------------------------
+stripped = []  # service names whose host port publishing we drop
+for svc, spec in services.items():
+    spec = spec or {}
+    ports = spec.get("ports") or []
+    if any((p or {}).get("published") for p in ports):
+        stripped.append(svc)
+
+# ---- Emit the override YAML (hand-rolled; no yaml dependency) --------------
+def y(s):
+    # Quote scalars that YAML could misread; safe for our name/key charset.
+    return json.dumps(s)
+
+lines = []
+lines.append("# GENERATED by gen-compose-worktree-override.sh — do not edit by hand.")
+lines.append("# Out-of-tree compose override that neutralizes the isolation leaks")
+lines.append("# COMPOSE_PROJECT_NAME does not cover (external / fixed-name volumes,")
+lines.append("# fixed host port bindings). Wire in without touching tracked files via:")
+lines.append(f"#   COMPOSE_FILE={base}:{out_abs}   (in the worktree's .env)")
+lines.append(f"# Source project: {project}   Worktree: {wt_name}")
+if shared_leaky:
+    for key, reason in shared_leaky:
+        lines.append(f"# KEPT SHARED (caller --share-volume): {key} ({reason})")
+lines.append("")
+
+if stripped:
+    lines.append("services:")
+    for svc in sorted(stripped):
+        lines.append(f"  {y(svc)}:")
+        # `!override []` REPLACES the base list (plain [] would append/merge).
+        lines.append("    ports: !override []")
+    lines.append("")
+
+if forked:
+    lines.append("volumes:")
+    for key, _reason, pinned in forked:
+        lines.append(f"  {y(key)}:")
+        lines.append(f"    name: {y(pinned)}")
+        lines.append("    external: false")
+    lines.append("")
+
+if not stripped and not forked:
+    lines.append("# No isolation leaks detected: every volume is already namespaced by")
+    lines.append("# COMPOSE_PROJECT_NAME and no service publishes a fixed host port.")
+    lines.append("# This override is a valid no-op; wiring it in is harmless.")
+    lines.append("")
+
+with open(out, "w") as f:
+    f.write("\n".join(lines).rstrip("\n") + "\n")
+
+# ---- Report to stderr (human + agent readable) ----------------------------
+def err(msg):
+    print(msg, file=sys.stderr)
+
+err(f"gen-compose-worktree-override: wrote {out}")
+err(f"  base compose file: {base}   project: {project}")
+if forked:
+    err("  forked volumes (were shared → now worktree-namespaced, non-external):")
+    for key, reason, pinned in forked:
+        err(f"    - {key} ({reason}) -> {pinned}")
+else:
+    err("  forked volumes: none")
+if shared_leaky:
+    err("  KEPT SHARED (caller --share-volume) — verify these are read-only caches:")
+    for key, reason in shared_leaky:
+        err(f"    - {key} ({reason})")
+if stripped:
+    err(f"  stripped host-port publishing from: {', '.join(sorted(stripped))}")
+else:
+    err("  stripped host-port publishing from: none")
+err(f"  wire in: COMPOSE_FILE={base}:{out_abs}  (append to the worktree's copied .env)")
+
+# stdout = override path ONLY (safe for $(...) capture).
+print(out_abs)
+PY

package/skills/vinta-derive-skills/resources/write-unit-test-packs/runners/jest.md

@@ -0,0 +1,49 @@
+# Jest runner pack
+
+Test-runner idioms for Jest. Stack-agnostic — framework-client and domain-mock specifics live in the **stack pack**, loaded alongside this one. The Testing Library / DOM notes apply only when the project renders UI. Read with the skill's universal rules. (Jest and Vitest share most idioms; the differences are noted.)
+
+## Structure & style
+
+- **`describe` / `it` (or `test`) with the behavior in the name.** `it("returns zero for an empty cart")`. The string is the spec.
+- **Assert full values with the right matcher.** `expect(result).toEqual(expected)` for deep value equality, `toBe` for primitives/identity, `toStrictEqual` when `undefined` fields / class identity matter. Avoid `toBeTruthy()` / `toBeDefined()` as the *only* assertion. For arrays assert contents (and order when it matters), not just `.toHaveLength(n)`.
+- Error paths: `expect(fn).toThrow(Message)` — assert the error, not just that it threw.
+
+## Table-drive instead of looping
+
+- Use `it.each([...])` / `describe.each` — each row is its own test with a literal expected value. Never loop inside one test recomputing the expectation.
+
+## Test data
+
+- **Prefer small factory/builder helpers over inline object literals** when the project has them — `makeUser({ email })` returns a valid object with only the relevant field overridden. Keep overrides minimal.
+- No factory convention → a local `base` object with per-test spread beats copy-pasted full literals.
+
+## Mocking & externals
+
+- **Mock only genuine externals, at the boundary.** `jest.mock("./module")`, `jest.fn()` for callbacks, `jest.spyOn(obj, "method")` for one method. Don't mock the unit under test or pure helpers.
+- **Reset between tests.** Set `clearMocks: true` (and consider `restoreMocks: true`) in the Jest config, or `afterEach(() => jest.restoreAllMocks())`. Stale mock state between tests is a top flake source.
+- **HTTP:** use **MSW** (`setupServer`) to intercept at the network layer and return canned responses — the real client path runs. Never let a real request out. Prefer MSW over hand-mocking `fetch`/`axios`.
+- **Complex APIs (e.g. LLM endpoints):** when hand-writing responses is impractical, record real traffic once and replay it from a committed cassette — [PollyJS](https://github.com/Netflix/pollyjs) or `nock`'s `nock.back`. Scrub secrets/PII from the recording, commit it, and never re-hit the live API in CI. Re-record deliberately when the contract changes.
+- **Time:** `jest.useFakeTimers()` + `jest.setSystemTime(...)`, `jest.useRealTimers()` in teardown. Don't assert on `Date.now()` live.
+- Prefer dependency injection over `jest.mock` when practical — fewer module-graph surprises.
+
+## UI / component tests
+
+- When the project renders UI, the component-testing idioms (Testing Library queries, `user-event`, asserting output not internals) live in the **React stack pack** — follow it there. Pure-logic units don't need it.
+
+## Isolation
+
+- Keep test files independent; don't share mutable module-level state. Clean up any DOM/global you touched in `afterEach`.
+- DB-backed tests: wrap each in a transaction rolled back in `afterEach`, or reset to a known seed. Never leave rows behind.
+
+## Jest-specific notes
+
+- `jest.mock(...)` is **hoisted** above imports; a mock factory can't reference outer variables unless they're prefixed `mock`. Return values from inside the factory.
+- Automock is off by default — keep it off; explicit mocks are clearer.
+- `jest.mock("module")` replaces the whole module for the file; use `jest.requireActual` to keep the real parts you don't mean to stub.
+
+## Pitfalls
+
+- Asserting on a mock's return you just configured — tests the mock, not the code.
+- Forgetting `await` on async expectations (`await expect(p).resolves...`) — the assertion never runs; the test passes hollow.
+- Snapshot sprawl: giant `toMatchSnapshot()` blobs get blindly re-recorded. Assert the fields that matter by value.
+- `toEqual` ignores `undefined` properties — use `toStrictEqual` when their presence/absence is part of the contract.

package/skills/vinta-derive-skills/resources/write-unit-test-packs/runners/pytest.md

@@ -0,0 +1,36 @@
+# pytest runner pack
+
+Test-runner idioms for pytest. Stack-agnostic — DB isolation and framework-client/mock specifics live in the **stack pack** (Django / FastAPI / Flask / …), loaded alongside this one. Read with the skill's universal rules.
+
+## Structure & style
+
+- **Plain functions + fixtures, not `unittest.TestCase`.** `def test_x():` with fixtures is the idiomatic form. Use a plain `class TestThing:` only when it genuinely groups (shared parametrization, a natural noun) — never subclass `TestCase` in new pytest code.
+- **Fixtures over `setUp`/`tearDown`.** Fixtures compose, scope (`function`/`module`/`session`), and `yield` for teardown. Put shared ones in `conftest.py`.
+- **Plain `assert`, not `self.assertEqual`.** pytest rewrites `assert` to show both sides. `assert result == expected`. Unordered collections: compare `sorted(...)` or sets. Approximate floats: `pytest.approx`.
+
+## Parametrize instead of looping
+
+- `@pytest.mark.parametrize("input,expected", [...])` — each row is its own test with a **literal** expected value. Add `ids=` when the repr is opaque. Never loop inside one test recomputing the expectation.
+
+## Test data
+
+- **Prefer `factory_boy` factories over inline dicts** when the project has them — `UserFactory(email="x@y.z")` spells out only the relevant field and stays correct as the model grows. `Factory.build()` (no persistence) vs `Factory.create()` (persisted). Override only the fields the assertion depends on.
+- No factories → a small local constructor/helper beats hand-assembled deep dicts repeated across tests.
+
+## Mocking & externals
+
+- **`mocker` (`pytest-mock`)** — `mocker.patch("module.where_its_used.thing")`, patching where the name is *looked up*, not where it's defined. Auto-undone at teardown.
+- **HTTP:** `responses` (requests) / `respx` (httpx) — assert the outbound request, return a canned response. `pytest-socket` (`--disable-socket`) is a good CI guard against stray real calls.
+- **Complex APIs (e.g. LLM endpoints):** when hand-writing responses is impractical, record real traffic once and replay it from a committed cassette — `vcrpy` (`@pytest.mark.vcr`) or `betamax`. Scrub secrets/PII from the cassette, commit it, and never re-hit the live API in CI. Re-record deliberately when the contract changes.
+- **Time/randomness:** `freezegun` (`@freeze_time`) or inject a clock; seed or patch `random`.
+- Don't patch your own pure functions or the unit under test — that tests the mock.
+
+## DB isolation
+
+- Wrap each DB-touching test in a transaction rolled back at teardown, via a fixture — the exact mechanism is stack-specific (see the stack pack). Never `DELETE FROM` in setup as a cleanup crutch, and never depend on test ordering.
+
+## Pitfalls
+
+- Patching the definition site instead of the SUT's namespace (`from lib import thing` → patch `sut.thing`) — the patch silently does nothing.
+- `assert mock.called` as the only assertion — asserts an interaction, not a result.
+- Comparing objects without value equality — give them a dataclass `__eq__` or assert the fields you mean.

package/skills/vinta-derive-skills/resources/write-unit-test-packs/runners/vitest.md

@@ -0,0 +1,43 @@
+# Vitest runner pack
+
+Test-runner idioms for Vitest. Stack-agnostic — framework-client and domain-mock specifics (e.g. Medplum's `MockClient`) live in the **stack pack**, loaded alongside this one. The Testing Library / DOM notes apply only when the project renders UI. Read with the skill's universal rules.
+
+## Structure & style
+
+- **`describe` / `it` (or `test`) with the behavior in the name.** `it("returns zero for an empty cart")`. The string is the spec; the failure line should read as a sentence.
+- **Assert full values with the right matcher.** `expect(result).toEqual(expected)` for deep value equality (objects/arrays), `toBe` for primitives/identity, `toStrictEqual` when `undefined` fields and class identity matter. Avoid `toBeTruthy()` / `toBeDefined()` as the *only* assertion — a wrong-but-truthy value passes. For arrays assert contents (and order when it matters), not just `.toHaveLength(n)`.
+- Use `expect(fn).toThrow(Message)` for error paths; assert the error, not just that it threw.
+
+## Table-drive instead of looping
+
+- Use `it.each([...])` / `describe.each` to cover many cases — each row is its own test with a literal expected value. Never `for`-loop inside one test recomputing the expectation.
+
+## Test data
+
+- **Prefer small factory/builder helpers over inline object literals** when the project has them — `makeUser({ email })` returns a valid object with only the relevant field overridden, and survives new required fields on the type. Keep overrides minimal: set only what the assertion depends on.
+- When there's no factory convention, a local `const base = {...}` with per-test spread (`{ ...base, qty: 2 }`) beats copy-pasting full literals across tests.
+
+## Mocking & externals
+
+- **Mock only genuine externals, at the boundary.** `vi.mock("./module")` for a whole module, `vi.fn()` for a passed-in callback, `vi.spyOn(obj, "method")` for one method. Don't mock the unit under test or pure helpers that run fine.
+- **Reset between tests.** Set `test: { clearMocks: true }` (or `restoreMocks: true`) in the Vitest config, or `afterEach(() => vi.restoreAllMocks())` — stale mock state leaking between tests is a top flake source.
+- **HTTP:** use **MSW** (`setupServer`) to intercept at the network layer and return canned responses — the code under test runs its real fetch/axios path. Never let a real request out. Prefer MSW over mocking `fetch` directly (which couples to the client).
+- **Complex APIs (e.g. LLM endpoints):** when hand-writing responses is impractical, record real traffic once and replay it from a committed cassette — [PollyJS](https://github.com/Netflix/pollyjs) or `nock`'s `nock.back`. Scrub secrets/PII from the recording, commit it, and never re-hit the live API in CI. Re-record deliberately when the contract changes.
+- **Time:** `vi.useFakeTimers()` + `vi.setSystemTime(...)`, and `vi.useRealTimers()` in teardown. Don't assert on `Date.now()` live.
+- **Modules with side effects on import:** prefer dependency injection over `vi.mock` when you can — fewer module-graph surprises.
+
+## UI / component tests
+
+- When the project renders UI, the component-testing idioms (Testing Library queries, `user-event`, asserting output not internals) live in the **React stack pack** — follow it there. Pure-logic units don't need it.
+
+## Isolation
+
+- Vitest isolates test files by default; keep it. Don't share mutable module-level state between tests. Clean up any DOM/global you touched in `afterEach`.
+- For DB-backed integration tests, wrap each test in a transaction rolled back in `afterEach`, or reset to a known seed — never leave rows behind.
+
+## Pitfalls
+
+- Asserting on a mock's return you just configured — tests the mock, not the code.
+- `vi.mock` factory hoisting surprises: `vi.mock` is hoisted above imports; reference values via the factory, not outer closures, or use `vi.hoisted`.
+- Forgetting `await` on async expectations (`expect(promise).resolves...`) — the assertion never runs and the test passes hollow.
+- Snapshot sprawl: giant `toMatchSnapshot()` blobs get blindly re-recorded. Assert the fields that matter by value.

package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/django.md

@@ -0,0 +1,36 @@
+# Django stack pack
+
+Django-specific testing — DB isolation, ORM data, settings, the test client, external boundaries. Loaded **only when the project is Django**, alongside the runner pack (pytest or unittest). Read with the skill's universal rules.
+
+## DB isolation & base class
+
+- **`TestCase` (or pytest-django's `db` fixture) wraps each test in a transaction rolled back at teardown** — the default, fast path, zero manual cleanup.
+- **`TransactionTestCase` / `@pytest.mark.django_db(transaction=True)`** truncates instead of rolling back — use **only** when the code commits its own transactions, tests `on_commit` hooks, or uses `select_for_update`. Markedly slower; not the default.
+- **`SimpleTestCase`** forbids DB access — use it to prove a unit is DB-free.
+- Never `Model.objects.all().delete()` in setup as a cleanup crutch; rely on rollback.
+
+## Test data
+
+- **Prefer `factory_boy` or `model_bakery` recipes over fixture files or inline `Model.objects.create(...)` walls** — `UserFactory(is_staff=True)` builds a valid instance spelling out only the relevant field. Fixture files drift from the schema silently.
+- With `TestCase`, put shared read-only data in **`setUpTestData(cls)`** (runs once per class inside the outer transaction), not `setUp` (per test). Only the fields the assertion needs.
+
+## Assertions
+
+- `self.assertEqual(response.json(), {...})` / `assertEqual(obj.status, "paid")` — full value, not `assertTrue(obj)` or a lone `Model.objects.count()`.
+- Use the specific ones for signal: `assertQuerySetEqual`, `assertContains`/`assertNotContains` (body + status), `assertRaisesMessage`, `assertNumQueries` (guard a hot path).
+- **`obj.refresh_from_db()` before asserting persisted state** — the in-memory instance hides DB defaults, signals, `auto_now`.
+
+## Externals
+
+- **`unittest.mock.patch`** where the name is *used* (`patch("myapp.services.stripe_client")`), not where defined.
+- **Email:** assert on `django.core.mail.outbox` (the test runner's `locmem` backend), don't mock `send`.
+- **HTTP:** `responses` / `respx` — never a real outbound call.
+- **Settings/time:** `@override_settings(...)` / `self.settings(...)`; `freezegun` or an injected clock.
+- **Test client:** `self.client` exercises the real URL/middleware/view stack — good. Mock only the external service the view calls, not the view.
+
+## Pitfalls
+
+- `TransactionTestCase` "to be safe" — slower for no reason. Default to `TestCase`.
+- `on_commit` hooks never fire under `TestCase` (the transaction rolls back) — use `TestCase.captureOnCommitCallbacks(execute=True)` or `TransactionTestCase`.
+- A lone `assertEqual(Model.objects.count(), 1)` — asserts a count, not that the right object with the right fields exists.
+- Asserting on `obj` after `save()` without `refresh_from_db()`.

package/skills/vinta-derive-skills/resources/write-unit-test-packs/stacks/fastapi.md

@@ -0,0 +1,34 @@
+# FastAPI stack pack
+
+FastAPI-specific testing — the test client, dependency overrides, async, DB isolation. Loaded **only when the project is FastAPI**, alongside the runner pack (usually pytest). Read with the skill's universal rules.
+
+## Test client & dependencies
+
+- **`TestClient(app)`** (Starlette, sync) exercises the real routing/validation/response stack. For async-native tests use **`httpx.AsyncClient(transport=ASGITransport(app=app))`**.
+- **Override dependencies with `app.dependency_overrides[dep] = fake`**, not by monkeypatching internals — inject a test DB session, a stubbed auth principal, a fake external client. Clear overrides in teardown (a fixture that yields then pops).
+- Don't mock the endpoint function; drive it through the client and override only what crosses the boundary.
+
+## Async
+
+- Use `pytest-asyncio` (`@pytest.mark.asyncio`) or `anyio`. Await the real coroutine; don't wrap sync mocks around async code. Mock async collaborators with `AsyncMock`.
+
+## DB isolation
+
+- Inject the DB session via a dependency and override it with a **session bound to a transaction rolled back in teardown** (SQLAlchemy: connection → `begin()` → session → rollback). Each test starts clean; no manual delete.
+- Prefer a real (SQLite/Postgres) test DB over mocking the ORM — mocking the session tests the mock.
+
+## Externals
+
+- **HTTP:** `respx` (httpx) / `responses` (requests) — never a real outbound call.
+- **Settings:** override via a `Settings` dependency or env fixture, not by mutating a global.
+- **Time/randomness:** `freezegun` or an injected clock; seed `random`.
+
+## Assertions
+
+- Assert `response.status_code` **and** `response.json()` by full value — not just the status. Validate against the response model's shape, not a lone key.
+
+## Pitfalls
+
+- Forgetting to clear `app.dependency_overrides` — leaks a fake into the next test.
+- Mixing sync `TestClient` with async DB drivers — use `AsyncClient` + async session instead.
+- Asserting only the status code while the body is wrong.