vinta-ai-workflows 0.2.0 → 0.3.0

package/CHANGELOG.md

@@ -5,6 +5,89 @@ All notable changes to `vinta-ai-workflows` are documented in this file.
 The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
 the project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.3.0] — 2026-07-17
+
+### Fixed
+
+- **`prepare-worktree` — data-corruption bug: `COMPOSE_PROJECT_NAME` alone does
+  NOT isolate a per-worktree docker-compose stack, so worktrees could run a
+  second database server against the main checkout's data volume.** Step 4
+  assumed `COMPOSE_PROJECT_NAME` fully isolates a compose stack. It does not for
+  two very common patterns: (1) volumes declared `external: true` and/or pinned
+  to a fixed top-level `name:` — the project name namespaces containers and
+  networks but NOT these volumes, so every worktree's `db` service mounted the
+  **same physical volume**; and (2) fixed host port bindings
+  (`ports: - "5432:5432"`) — published to the same host port across compose
+  projects. On a Django + docker-compose project (2026-07-17) this ran two
+  Postgres postmasters against one PGDATA for a whole test run; when the
+  worktree stack was `docker compose down`'d, its Postgres removed the shared
+  `postmaster.pid` and the **main checkout's database self-terminated**
+  (recovered on restart — luck, not safety). The fix also corrects a deeper
+  conceptual error: Step 3a's "share the DB (don't fork)" is now explicit that
+  *share* means share a **connection to a single already-running server** — for
+  a compose-delivered DB the data volumes are **always forked, independent of
+  `schema_change` / `test_db_strategy`**, because the worktree runs its own
+  server and two servers on one volume is corruption.
+
+  **User impact / migration:** existing worktrees provisioned before this fix
+  are **unsafe to boot their DB compose stack while the main stack is up** —
+  they share main's data volume. For an already-provisioned worktree, regenerate
+  the override without re-running the whole skill:
+
+  ```bash
+  ai-tools/skills/prepare-worktree/scripts/gen-compose-worktree-override.sh \
+    --main <main-checkout-root> --worktree <worktree-path> --name <name> \
+    --out <summary_dir>/<name>.docker-compose.override.yml
+  echo "COMPOSE_FILE=<base-compose-file>:<override-path>" >> <worktree>/.env
+  ```
+
+  Then confirm the worktree's resolved DB volume name differs from main's before
+  booting its stack. **Consumers**: re-sync (`vinta-sync-ai-tools`) to pick up
+  the new script, the reworked Step 4/Step 3a, and the `shared_volumes` config.
+
+### Added
+
+- **`prepare-worktree` bundled script
+  `scripts/gen-compose-worktree-override.sh`.** Detects the compose isolation
+  leaks above entirely from `docker compose config --format json` (no project /
+  service / volume name hardcoded) and emits a **generated, out-of-tree**
+  compose override that re-pins every `external:`/fixed-`name:` volume to a
+  non-external, worktree-namespaced volume (`<project>_wt_<name>_<volkey>`) and
+  strips fixed host-port publishing (via compose's `!override []` — a plain
+  `ports: []` merges/appends in Compose v2+; verified on Docker Compose v5.1.1).
+  The override is written under `<summary_dir>/` and wired in via
+  `COMPOSE_FILE=<base>:<override>` appended to the worktree's **copied** `.env`,
+  so **no tracked compose file is ever mutated**. Volumes safe to keep shared
+  (a read-only dep/venv cache) are exempted via `--share-volume`.
+- **`skills.prepare-worktree.shared_volumes` config field** (array of compose
+  volume keys, default `[]`). Lists volumes safe to keep shared across worktrees
+  — read-only dep/venv caches only, never a data/DB volume. Feeds
+  `gen-compose-worktree-override.sh --share-volume`. Added to the config schema
+  and the bootstrap template's `prepare-worktree` block.
+- **`prepare-worktree` Step 4a "Neutralize compose isolation leaks"**, a
+  Verification invariant that boots the worktree's DB service and asserts its
+  resolved data volume name differs from main's (failing provisioning loudly
+  otherwise), a matching Pitfall + Rules entry, and a `state.compose` summary
+  schema extension (per-volume fork decisions, override path, stripped ports)
+  so teardown is mechanical (`docker volume rm` the forked names; `rm` the
+  override — never `down -v`, which would nuke shared volumes).
+
+### Changed
+
+- **`prepare-worktree` foundation skill — rebalanced Step 3b (test database)
+  to be stack-agnostic.** The section previously led with a detailed
+  `pytest-django` recipe (with `vitest`/`jest` and Rails as one-liners),
+  which read as Django-centric. It now opens with the stack-agnostic
+  principle — give the worktree its own test DB name, inject it through
+  whatever channel the runner already reads (env var or per-worktree config
+  override), and never edit tracked test config in place — then lists the
+  runners as equal-weight illustrations grouped by *how they take config*:
+  env-var runners (`vitest`/`jest`, `go test`, `cargo test`), config-file
+  runners (`pytest-django`), convention-based runners (Rails), and
+  compose-based test DBs. No behavior change; the pytest-django mechanics
+  (`conftest_worktree.py`, don't touch the tracked `conftest.py`) are
+  preserved. **Consumers**: re-sync to pick up the reworded skill body.
+
 ## [0.2.0] — 2026-07-15
 
 ### Added

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vinta-ai-workflows",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Bootstrap AI tooling (AGENTS.md, skills, sub-agents) across Claude Code, Codex, Cursor, and VS Code + GitHub Copilot.",
   "type": "module",
   "bin": {

package/schemas/vinta-ai-workflows-config.v1.schema.json

@@ -420,10 +420,16 @@
               "default": "fork-on-schema-change"
             },
             "summary_dir": {
-              "description": "Where per-worktree summary YAMLs land. Default `.vinta-ai-workflows/worktrees` — covered by the umbrella gitignore added in 0.1.3. The skill writes `<summary_dir>/<name>.yaml` per provisioned worktree; teardown reads it.",
+              "description": "Where per-worktree summary YAMLs land. Default `.vinta-ai-workflows/worktrees` — covered by the umbrella gitignore added in 0.1.3. The skill writes `<summary_dir>/<name>.yaml` per provisioned worktree; teardown reads it. The generated compose override (see `shared_volumes`) also lands here as `<summary_dir>/<name>.docker-compose.override.yml`.",
               "type": "string",
               "minLength": 1,
               "default": ".vinta-ai-workflows/worktrees"
+            },
+            "shared_volumes": {
+              "description": "Compose volume KEYS (as they appear under top-level `volumes:` in the project's compose config) that are safe to keep SHARED across worktrees — a read-only dep/venv/asset cache with no write churn. The skill's compose-isolation step (Step 4a) forks EVERY `external:`/fixed-`name:` volume into a worktree-namespaced copy by default (COMPOSE_PROJECT_NAME does not isolate those, and two servers on one DATA volume corrupts the store); volumes listed here are exempted and stay shared. NEVER list a database/data volume here. Passed through to `scripts/gen-compose-worktree-override.sh --share-volume`. Empty/omitted = fork every leaky volume (safest).",
+              "type": "array",
+              "items": { "type": "string", "minLength": 1 },
+              "default": []
             }
           }
         },

package/skills/vinta-bootstrap-ai-tools/SKILL.md

@@ -312,6 +312,10 @@ skills:
     compose_network: per-worktree
     test_db_strategy: <prepare-worktree follow-up — fork-on-schema-change | always-fork | share>
     summary_dir: .vinta-ai-workflows/worktrees
+    # Compose volume keys safe to keep shared across worktrees (read-only dep/venv
+    # caches only — NEVER a data/DB volume). Empty = fork every external/named
+    # volume (safest). See prepare-worktree Step 4a.
+    shared_volumes: []
 
   # Only emit this block when foundation_skills.add-one-off-script = enabled.
   add-one-off-script:

package/skills/vinta-derive-skills/resources/foundation-skills/prepare-worktree/SKILL.md

@@ -109,6 +109,8 @@ Default: **copy** for pnpm + npm + cargo + go; **reinstall** for poetry + venv +
 
 `env_change = true` → `cp <main>/.env <worktree>/.env`. Then mutate per the **Database fork** step (DB URL) + the **Docker / compose isolation** step (compose project name).
 
+**Compose forces a copy.** If the project uses compose, the [4a](#4a--neutralize-compose-isolation-leaks) sub-step appends `COMPOSE_FILE=…` to the worktree's `.env` — a mutation. So a compose project ALWAYS needs a copied `.env` (or a copied `.envrc`) even when `env_change = false`; a symlinked `.env` would leak the `COMPOSE_FILE` line into main. Treat "project uses compose" as an independent reason to copy `.env`.
+
 `.env.example` is **always** symlinked — it's committed; the worktree should keep tracking it.
 
 ### 2c — Skip rules
@@ -129,6 +131,11 @@ Whatever the app reads + writes during local dev. Detect the connection string s
 2. Project config files: `config/database.yml` (Rails), `database.ini`, `prisma/schema.prisma`'s `datasource`, `knexfile.js`, Django `settings.py`'s `DATABASES`.
 3. Docker compose service env: `services.db.environment.POSTGRES_DB`.
 
+**First, determine HOW the DB is delivered — this changes what "share" even means.** Two deliveries:
+
+- **External server** — the DB runs on a server the worktree only *connects* to (a system Postgres, an RDS instance, a compose service the worktree does NOT boot its own copy of). Here "share" = point the worktree's connection string at the one already-running server. Fork = clone into a new DB *on that same server*.
+- **Per-worktree compose stack** — the DB is delivered *through the worktree's own `docker compose up`* (a `db` service the worktree boots). Here the worktree runs its **own server process**. "Share" the way the connection-string sense means is impossible without corruption: booting a second server against the same data volume is two postmasters on one PGDATA. **For compose-delivered DBs the data volumes are ALWAYS forked, independent of `schema_change`** — see the [Neutralize compose isolation leaks](#4a--neutralize-compose-isolation-leaks) sub-step, which does the volume fork mechanically. `schema_change = false` may still let you skip *cloning main's data* into the forked volume (boot empty + migrate), but it never authorizes a shared data volume.
+
 For each detected DB, ask the user (`AskUserQuestion`) once:
 
 - **Fork the DB** — recommended when `schema_change = true` or when the plan's phases run destructive migrations. Strategy depends on the engine:
@@ -137,7 +144,7 @@ For each detected DB, ask the user (`AskUserQuestion`) once:
   - **SQLite (file-based)** — `cp <main>/db.sqlite3 <worktree>/db.sqlite3`. Symlink would defeat the point.
   - **Mongo / Redis / Elasticsearch** — engine-specific clone, OR per-worktree DB name / key prefix (`REDIS_URL=redis://localhost:6379/<index>` with a free index; `MONGODB_URI=mongodb://localhost:27017/<db>_wt_<name>`).
 
-- **Share the DB (don't fork)** — only safe when `schema_change = false` AND there's no risk of conflicting writes. Symlink the SQLite file or point the worktree's env at the same URL. Warn explicitly: "DB shared with main checkout — destructive ops here will be visible in main."
+- **Share the DB (don't fork)** — means share a *connection to a single already-running server*, nothing more. Only safe when the DB is an **external server** (NOT compose-delivered — see the delivery check above) AND `schema_change = false` AND there's no risk of conflicting writes. Symlink the SQLite file or point the worktree's env at the same URL. Warn explicitly: "DB shared with main checkout — destructive ops here will be visible in main." **Never read this bullet as permission to point a second per-worktree compose server at a shared data volume** — that is not sharing a connection, it is running two servers on one data directory, and it corrupts the store regardless of `schema_change`.
 
 - **Stub the DB** — point the worktree at a fresh empty DB (`createdb <main_db>_wt_<name>` + run all migrations + seed if a seed exists). Best when the feature needs schema parity but no production data.
 
@@ -147,10 +154,12 @@ Record the chosen strategy + the forked DB name in the worktree summary (written
 
 A different beast — tests on the same engine but a different DB name (`<main_db>_test`, `test_<repo>`). Parallel worktrees running tests against the same `<main_db>_test` will overwrite each other's fixtures and produce flaky failures.
 
-- **`pytest-django` / `pytest`** — set `--reuse-db` per worktree via a per-worktree `DJANGO_SETTINGS_MODULE` env, OR override `DATABASES['default']['NAME']` to `<main_db>_test_wt_<name>` in `conftest.py` when the env var `WORKTREE_NAME` is set. Drop a `conftest_worktree.py` patch in the worktree (don't edit `conftest.py` in tracked code — too easy to commit by accident).
-- **`vitest` / `jest`** — set `TEST_DATABASE_URL` per worktree; ensure the test setup respects it.
-- **Rails** — `DATABASE_URL` for the `test` env, `<main_db>_test_wt_<name>`.
-- **Docker-compose-based test DB** — see the **Docker / compose isolation** step below; per-worktree compose project name fixes it.
+**The principle (stack-agnostic):** give the worktree its own test DB name (`<main_db>_test_wt_<name>`), inject it through whatever channel the test runner already reads (an env var or a per-worktree config override), and **never edit tracked test config in place** — a worktree-local override or an env var is safe; a change to a committed config file is one `git add -A` away from leaking into main. How each runner exposes that channel:
+
+- **Env-var runners (`vitest` / `jest`, `go test`, `cargo test`, most CI-shaped suites)** — set `TEST_DATABASE_URL` (or the suite's equivalent) per worktree and confirm the test setup reads it. Simplest case; no file edits at all.
+- **Config-file runners (`pytest-django` / `pytest`)** — set `--reuse-db` per worktree via a per-worktree `DJANGO_SETTINGS_MODULE`, OR override `DATABASES['default']['NAME']` to `<main_db>_test_wt_<name>` guarded on a `WORKTREE_NAME` env var. Put the override in a `conftest_worktree.py` in the worktree — don't touch the tracked `conftest.py`.
+- **Convention-based runners (Rails)** — point the `test` env's `DATABASE_URL` at `<main_db>_test_wt_<name>`.
+- **Docker-compose-based test DB (any stack)** — see the **Docker / compose isolation** step below; a per-worktree compose project name fixes it.
 
 If `test_infra_change = true` → fork the test DB unconditionally. If `false` and the user says "share is fine" → still flag the race risk; offer a one-line fix to switch later.
 
@@ -166,10 +175,15 @@ If the project has a `docker-compose.yml` / `compose.yaml` / `docker-compose.ove
 
 1. Set `COMPOSE_PROJECT_NAME=<repo>_<worktree-name>` in the worktree's `.env` (or in a `.envrc` if the project uses direnv). Then `docker compose up` inside the worktree spins a fresh container set.
 
-2. **Shared network** — if the worktree's app needs to reach services running in the *main* checkout's compose stack (a queue, a cache, a search index that's expensive to spin twice), put them on an external network:
+   **`COMPOSE_PROJECT_NAME` is necessary but NOT sufficient.** It namespaces containers and networks, and auto-named volumes — but it does **not** isolate two very common compose patterns: volumes declared `external: true` and/or pinned to a fixed top-level `name:`, and services with fixed host port bindings (`ports: - "5432:5432"`). A shared *database* volume is the dangerous one: two worktree stacks (or a worktree stack + main's stack) mount the same physical volume, so two server processes run against one data directory → corruption. The next sub-step closes both leaks mechanically.
+
+2. **Neutralize compose isolation leaks** — see [4a](#4a--neutralize-compose-isolation-leaks) below. Run this whenever the project has compose, before ever booting the worktree's stack.
+
+3. **Shared network** — if the worktree's app needs to reach services running in the *main* checkout's compose stack (a queue, a cache, a search index that's expensive to spin twice), put them on an external network:
 
    ```yaml
-   # docker-compose.override.yml in the worktree (or in main; pick one place)
+   # Add to the SAME out-of-tree override the 4a sub-step generates — NOT a
+   # tracked docker-compose.override.yml at the worktree root (see 4a for why).
    networks:
      shared:
        external: true
@@ -182,9 +196,44 @@ If the project has a `docker-compose.yml` / `compose.yaml` / `docker-compose.ove
 
    Or use `network_mode: host` if the project uses that pattern. Decide once per project and record in `.vinta-ai-workflows.yaml` → `run_options.prepare-worktree.compose_network: per-worktree | shared-external | host`.
 
-3. **Linters / formatters / test runners that run inside docker** (a Dockerfile-based `pnpm test`, a `lint` target that mounts source into a sidecar container) — these MUST run with the worktree's `COMPOSE_PROJECT_NAME`. Otherwise concurrent runs hit the same container and one wins. The standard fix: ensure the project's lint/test scripts read `COMPOSE_PROJECT_NAME` from env instead of hard-coding it.
+4. **Linters / formatters / test runners that run inside docker** (a Dockerfile-based `pnpm test`, a `lint` target that mounts source into a sidecar container) — these MUST run with the worktree's `COMPOSE_PROJECT_NAME`. Otherwise concurrent runs hit the same container and one wins. The standard fix: ensure the project's lint/test scripts read `COMPOSE_PROJECT_NAME` from env instead of hard-coding it.
+
+5. **Per-worktree volumes** — for a volume the project leaves un-named, compose auto-namespaces it by project name and it's already isolated. But projects routinely pin data volumes with `external: true` or a fixed `name:` (for good reasons of their own — you must NOT edit their committed compose to "fix" that). Those are the leaks the [4a](#4a--neutralize-compose-isolation-leaks) sub-step re-pins in a generated override. Never touch the tracked compose file to change a volume's isolation; the override is the only lever.
+
+### 4a — Neutralize compose isolation leaks
+
+Runs whenever the project has any compose file, before the worktree's stack is ever booted. Fully project-agnostic — every decision is driven by `docker compose config --format json`, no project / service / volume name is hardcoded.
+
+**1. Generate the override.** Run the bundled [scripts/gen-compose-worktree-override.sh](scripts/gen-compose-worktree-override.sh):
+
+```bash
+override=$(scripts/gen-compose-worktree-override.sh \
+  --main     <main-checkout-root> \
+  --worktree <worktree-path> \
+  --name     <worktree-name> \
+  --out      <summary_dir>/<name>.docker-compose.override.yml \
+  --share-volume <volkey>...)   # 0+ volumes safe to keep shared (see below)
+```
+
+The script renders the resolved config from the main checkout and, for the worktree:
+- re-pins every volume that is `external: true` OR carries a fixed top-level `name:` to a **non-external, worktree-namespaced** volume (`<project>_wt_<name>_<volkey>`), so the worktree's server writes to its OWN physical volume;
+- strips host port publishing from every service that publishes a fixed port (via compose's `!override []` — a plain `ports: []` **merges/appends** in Compose v2+ and would NOT strip it; verified on Docker Compose v5.1.1);
+- leaves alone any volume already namespaced by project name (auto-named), and any volume the caller passes via `--share-volume` (a read-only dep/venv cache with no dep churn is safe to keep shared — the config's `shared_volumes` list feeds this arg).
+
+The script exits non-zero with a clear message if `docker compose` is unavailable or the config doesn't parse; surface that and stop (don't boot a stack you couldn't prove is isolated).
+
+**2. Write it OUT OF TREE and wire it in via the copied `.env`.** The override goes to `<summary_dir>/<name>.docker-compose.override.yml` (under the already-ignored `.vinta-ai-workflows/` umbrella) — **NOT** a `docker-compose.override.yml` at the worktree root. Rationale: a root `docker-compose.override.yml` is auto-loaded by compose, but it is frequently a **tracked** file (it was in the project that hit the 2026-07-17 incident); writing the generated override there pollutes the worktree diff and risks `git add -A` leaking it into main. Instead, wire it in by appending to the worktree's **copied** `.env`:
+
+```bash
+# The generator prints exactly this line (base:override) on stderr — use it verbatim.
+echo "COMPOSE_FILE=<base-compose-file>:<ABSOLUTE-override-path>" >> <worktree>/.env
+```
+
+`COMPOSE_FILE` makes compose load base + override for every `docker compose` call in the worktree without touching any tracked file. The **override path must be absolute** — it lives under `<summary_dir>` (outside the worktree), so a bare basename wouldn't resolve from the worktree's compose project dir; the generator prints the absolute path for you (and on stdout, for `$(...)` capture). The base file stays relative — the worktree carries its own tracked copy. **This requires `.env` to be COPIED, not symlinked** — appending to a symlinked `.env` leaks the line into main. If the **Env files** step decided `env_change = false` (symlink), flip `.env` to a copy here (compose isolation is itself an env change) and note it in the summary. Use the exact base compose filename the generator reports (`compose.yaml` vs `docker-compose.yml`).
+
+**3. Record the decision.** Write the per-volume fork decisions and the override path into the summary YAML's `state.compose` (see the **Write the summary file** step). The override's own `volumes:` block is the teardown manifest — each pinned `name:` is a `docker volume rm` target.
 
-4. **Per-worktree volumes** — let compose auto-create volumes namespaced by project name. Don't manually `external: true` data volumes (defeats isolation).
+**Relation to `test_db_strategy`.** The `test_db_strategy: fork-on-schema-change` config governs the **test-DB axis only** (Step 3b — which test DB *name* the runner targets). It says nothing about compose data volumes and must **never** be read as permission to share the compose stack's volumes. Volume forking here is unconditional for compose-delivered DBs, independent of `schema_change` and independent of `test_db_strategy`.
 
 ## Step 5 — Other shared infra to fork
 
@@ -303,6 +352,14 @@ state:
   compose:
     project_name: <repo>_<worktree-name>
     network_strategy: per-worktree | shared-external | host
+    override_path: <summary_dir>/<name>.docker-compose.override.yml   # null when no compose
+    base_compose_file: docker-compose.yml    # the base file COMPOSE_FILE prepends
+    forked_volumes:                          # [] when nothing leaked; teardown removes each name
+      - volume_key: pgdata                   # the compose key
+        reason: external | fixed-name        # why it leaked past COMPOSE_PROJECT_NAME
+        forked_name: <repo>_wt_<name>_pgdata # the worktree-namespaced physical volume
+    shared_volumes: [virtualenv]             # leaky volumes deliberately kept shared (--share-volume)
+    ports_stripped_from: [db, cache]         # services whose fixed host ports the override removed
   other:
     redis_db_index: <int>
     s3_prefix: wt-<name>/
@@ -346,7 +403,12 @@ guard is active; the orchestrator's post-run stray-write check is the backstop.
 When the plan is merged / abandoned:
   git worktree remove <path>
   <drop-db command>          # if fork strategy was `fork`
-  docker compose -p <project_name> down -v   # if compose was forked
+  # Compose (if forked). Do NOT use `down -v` — it can also nuke SHARED volumes
+  # (a fixed-name dep/venv cache). Bring the stack down, then remove only the
+  # worktree-namespaced volumes the override created:
+  docker compose -p <project_name> down
+  docker volume rm <each state.compose.forked_volumes[].forked_name>
+  rm <override_path>         # the generated out-of-tree compose override
 
 The summary file at `.vinta-ai-workflows/worktrees/<name>.yaml` records every
 decision for mechanical teardown.
@@ -371,7 +433,10 @@ A sibling skill / explicit command — not auto-run by this skill. Steps:
 1. Confirm no uncommitted changes in the worktree (`git -C <path> status`).
 2. `git worktree remove <path>` (or `--force` after explicit confirmation if the user is fine losing work).
 3. Drop the forked DB (`dropdb <forked_name>`, `DROP DATABASE`, `rm <sqlite-file>`, …) — read the strategy from the summary YAML.
-4. `docker compose -p <project_name> down -v` for forked compose.
+4. Compose teardown (read `state.compose` from the summary YAML):
+   - `docker compose -p <project_name> down` — **not `down -v`**; a bare `down -v` also removes shared volumes (a fixed-name dep/venv cache in `state.compose.shared_volumes`), which belong to main.
+   - `docker volume rm <forked_name>` for each entry in `state.compose.forked_volumes` — these are the worktree-only volumes the generated override created; removing them is safe and leaves main's / shared volumes untouched.
+   - `rm <state.compose.override_path>` — delete the generated out-of-tree override.
 5. Remove `.vinta-ai-workflows/worktrees/<name>.yaml`.
 
 Every step gated on user confirmation when the worktree has un-pushed branches.
@@ -380,6 +445,7 @@ Every step gated on user confirmation when the worktree has un-pushed branches.
 
 - **Symlink for reads, copy for writes, fork for state.** This is the only mental model that scales. Default to fork when unsure — disk is cheap, corrupted main-checkout DBs are not.
 - **Never share a writable DB across worktrees by default.** The race conditions are subtle and the failure mode is silent data corruption.
+- **`COMPOSE_PROJECT_NAME` does not isolate `external:`/fixed-`name:` volumes or fixed host ports — always run the [4a](#4a--neutralize-compose-isolation-leaks) generator.** For a compose-delivered DB the data volumes are forked *unconditionally* (independent of `schema_change` and `test_db_strategy`): the worktree runs its own server, and two servers on one volume is corruption. "Share the DB" only ever means share a connection to a single already-running server, never a second server on shared storage. The fix is a generated, out-of-tree override + a `COMPOSE_FILE` line in the copied `.env` — never an edit to any tracked compose file. Verify the volume-differs invariant before declaring the worktree runnable.
 - **Every fork decision lands in `.vinta-ai-workflows/worktrees/<name>.yaml`.** Teardown reads it; humans grep it; agents resuming a stalled plan read it. No decision lives only in conversation memory.
 - **Worktree root governed by runtime conventions.** claude-code uses `.claude/worktrees/`; other harnesses use sibling dirs. Don't fight the harness — match it.
 - **Don't mutate the main checkout from this skill.** Every write goes to the worktree or to `.vinta-ai-workflows/`. Forking a Postgres DB is the one exception (the new DB lives in the same server) — document it loudly in the summary.
@@ -395,6 +461,9 @@ Every step gated on user confirmation when the worktree has un-pushed branches.
 - **Symlinking `node_modules` for a `pnpm add` phase.** The new package writes back through the symlink into the main checkout's store. Detect dep churn in the **Plan inspection** step and copy/reinstall instead.
 - **Forking the dev DB but forgetting the test DB.** Tests still hit the shared test DB and stomp on parallel worktrees' fixtures. Both axes need their own decision in the **Database fork** step.
 - **Forgetting `COMPOSE_PROJECT_NAME`.** Containers from worktree-A overwrite worktree-B's containers; volumes get nuked. Set it in `.env` so every `docker compose` call inherits.
+- **Trusting `COMPOSE_PROJECT_NAME` to isolate `external:`/fixed-`name:` volumes and fixed host ports — it does NOT.** These defeat the project-name namespace: every worktree's compose stack mounts the SAME external/named volume and binds the SAME host port. For a DB volume that means two server processes on one data directory — the 2026-07-17 corruption incident (a worktree's `docker compose down` deleted the shared `postmaster.pid` and the main checkout's Postgres self-terminated). The fix is the [4a](#4a--neutralize-compose-isolation-leaks) generated out-of-tree override, never an edit to the tracked compose file. Verify the invariant (Verification step 6), don't assume it.
+- **Confusing "share the DB" (a connection to one running server) with "share the data volume" (a second server on shared storage).** The first can be safe when `schema_change = false`; the second is always corruption. `schema_change = false` and `test_db_strategy` never authorize a shared compose data volume — see [3a](#3a--dev--app-database).
+- **Writing the compose override to the worktree-root `docker-compose.override.yml`.** It's auto-loaded (convenient) but often a *tracked* file — the generated override then shows up in the diff and one `git add -A` leaks it into main. Write it under `<summary_dir>/` and wire it in via `COMPOSE_FILE` in the COPIED `.env` ([4a](#4a--neutralize-compose-isolation-leaks)).
 - **Sharing a Redis DB without per-worktree prefix.** Tests writing `user:123` collide across worktrees. Pick an index OR a key prefix.
 - **Copying `node_modules` for yarn PnP / absolute-path setups.** The copy carries baked-in paths from the main checkout. Reinstall instead — pnpm's relative-symlink store is the safe-to-copy exception.
 - **Forgetting to `--allow` the `.vinta-ai-workflows` dir under sandbox.** It's shared/symlinked but the run *writes* it (summary YAMLs, prs-context). If it's not an `--allow` exception, those writes hit the read-only main subtree and fail mid-run. Pass it alongside the worktree path.
@@ -413,4 +482,7 @@ After the **Write the summary file** step writes the summary:
 3. `git -C <worktree-path> status` is clean (no accidental file additions from the prep step).
 4. The summary YAML parses (`python3 -c "import yaml; yaml.safe_load(open('.vinta-ai-workflows/worktrees/<name>.yaml'))"`).
 5. `WORKTREE.md` exists at the worktree root with accurate fork / share annotations.
-6. Optional smoke test: run a single new-test command in the worktree (e.g. `pytest -x tests/health.py`) — confirms env vars resolved, DB reachable, deps importable.
+6. **Compose volume-isolation invariant (when the DB is compose-delivered).** This is the checked form of the footgun that caused the 2026-07-17 incident — do not skip it when the project boots its DB via compose. From the worktree:
+   - Resolve the worktree's DB data volume: `docker compose config --format json | <read the db service's data volume `source`, then that volume's `.name`>`. Assert it **differs** from the main checkout's resolved DB volume name (run the same read in `<main-checkout-root>`). If they're equal, the override didn't take — **fail provisioning loudly**; do not proceed.
+   - Boot only the DB service in the worktree (`docker compose up -d <db-service>`), then confirm the **main checkout's** DB container is still healthy/running afterward (`docker inspect`/`pg_isready` against main's DB). Bring the worktree DB back down. A main-DB disturbance here means isolation failed — fail provisioning.
+7. Optional smoke test: run a single new-test command in the worktree (e.g. `pytest -x tests/health.py`) — confirms env vars resolved, DB reachable, deps importable.

package/skills/vinta-derive-skills/resources/foundation-skills/prepare-worktree/scripts/gen-compose-worktree-override.sh

@@ -0,0 +1,211 @@
+#!/usr/bin/env bash
+# gen-compose-worktree-override.sh — generate an out-of-tree docker-compose
+# override that neutralizes the isolation leaks COMPOSE_PROJECT_NAME does NOT
+# cover, so a per-worktree compose stack cannot collide with the main checkout's.
+#
+# Why this exists: COMPOSE_PROJECT_NAME namespaces containers and networks, but
+# NOT two very common compose patterns —
+#
+#   1. Volumes declared `external: true` and/or pinned to a fixed top-level
+#      `name:`. These are addressed by name, so EVERY worktree's compose stack
+#      mounts the SAME physical volume. For a database volume that means two
+#      postmasters on one PGDATA → data corruption (the 2026-07-17 incident).
+#   2. Fixed host port bindings (`ports: - "5432:5432"`). These publish to a
+#      fixed host port that collides across compose projects regardless of name.
+#
+# The fix is a GENERATED, out-of-tree override (never a mutation of the project's
+# committed compose files) that, for the worktree:
+#   - re-pins every leaky volume to a NON-external, worktree-namespaced volume
+#     (`<project>_wt_<name>_<volkey>`), EXCEPT volumes the caller marks shareable
+#     (a read-only dep/venv cache with no dep churn is safe to keep shared);
+#   - strips host port publishing from every service that has it, using compose's
+#     `!override []` tag (a plain `ports: []` MERGES/appends in Compose v2+ and
+#     would NOT strip the port — verified on Docker Compose v5.1.1).
+#
+# Detection is 100% generic — no project / service / volume name is hardcoded.
+# Everything comes from `docker compose config --format json` run in the main
+# checkout. A volume leaks when `.external == true` OR its resolved `.name`
+# differs from the auto-namespaced form `<project>_<volkey>` (i.e. it carries a
+# fixed top-level `name:`). Setting `external: false` in the override is
+# mandatory — without it the base `external: true` merges through and the volume
+# stays externally-pinned (verified).
+#
+# Usage:
+#   gen-compose-worktree-override.sh \
+#     --main     <main-checkout-root> \
+#     --worktree <worktree-path> \
+#     --name     <worktree-name> \
+#     --out      <override-output-path> \
+#     [--share-volume <volkey>]...   # volumes safe to keep shared (repeatable)
+#
+# Output:
+#   stdout — the override path (only, on success) so it is safe to capture with
+#            $(...). stderr carries a human-readable decision report + the
+#            suggested `COMPOSE_FILE=<base>:<override>` wiring line.
+#   The generated override's `volumes:` block lists the forked volumes by their
+#   pinned name — that IS the teardown manifest (`docker volume rm <name>`).
+#
+# Exit: 0 on success; 2 on argument / setup errors (no `docker compose`, config
+#       does not parse, no base compose file found).
+
+set -u
+
+die() { echo "gen-compose-worktree-override: $*" >&2; exit 2; }
+
+MAIN=""; WORKTREE=""; NAME=""; OUT=""
+SHARE=()
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --main)         shift; [ $# -gt 0 ] || die "--main needs a path"; MAIN="$1"; shift ;;
+    --worktree)     shift; [ $# -gt 0 ] || die "--worktree needs a path"; WORKTREE="$1"; shift ;;
+    --name)         shift; [ $# -gt 0 ] || die "--name needs a value"; NAME="$1"; shift ;;
+    --out)          shift; [ $# -gt 0 ] || die "--out needs a path"; OUT="$1"; shift ;;
+    --share-volume) shift; [ $# -gt 0 ] || die "--share-volume needs a volume key"; SHARE+=("$1"); shift ;;
+    *)              die "unknown arg: $1" ;;
+  esac
+done
+
+[ -n "$MAIN" ]     || die "--main <main-checkout-root> required"
+[ -n "$WORKTREE" ] || die "--worktree <path> required"
+[ -n "$NAME" ]     || die "--name <worktree-name> required"
+[ -n "$OUT" ]      || die "--out <override-path> required"
+
+command -v docker >/dev/null 2>&1 || die "docker not found on PATH — cannot generate compose override"
+docker compose version >/dev/null 2>&1 || die "'docker compose' unavailable — is the Compose v2 plugin installed?"
+
+# Canonicalize the main checkout (must exist).
+[ -d "$MAIN" ] || die "main checkout path is not a directory: $MAIN"
+MAIN=$(cd "$MAIN" && pwd -P)
+
+# Locate the base compose file compose will auto-load (standard search order).
+BASE=""
+for f in compose.yaml compose.yml docker-compose.yaml docker-compose.yml; do
+  if [ -f "$MAIN/$f" ]; then BASE="$f"; break; fi
+done
+[ -n "$BASE" ] || die "no compose file (compose.yaml / docker-compose.yml) found in $MAIN"
+
+# Render the resolved config from the MAIN checkout. Do not force
+# COMPOSE_PROJECT_NAME: let it default exactly as the project normally resolves
+# it, so the auto-namespace comparison below reflects reality.
+CONFIG_JSON=$(cd "$MAIN" && docker compose config --format json 2>/dev/null) \
+  || die "'docker compose config' failed in $MAIN — the compose config does not parse"
+
+mkdir -p "$(dirname "$OUT")" || die "cannot create output dir for $OUT"
+
+CONFIG_JSON="$CONFIG_JSON" OUT="$OUT" NAME="$NAME" BASE="$BASE" \
+python3 - "${SHARE[@]:-}" <<'PY' || die "failed to generate override"
+import json, os, sys
+
+cfg = json.loads(os.environ["CONFIG_JSON"])
+out = os.environ["OUT"]
+wt_name = os.environ["NAME"]
+base = os.environ["BASE"]
+share = {s for s in sys.argv[1:] if s}
+
+# The override lives OUT of the worktree (under <summary_dir>), so the COMPOSE_FILE
+# entry must reference it by ABSOLUTE path — a basename would only resolve if the
+# file sat in the compose project dir. The base file is worktree-local (the
+# worktree carries its own tracked copy), so it stays relative.
+out_abs = os.path.abspath(out)
+
+project = cfg.get("name") or "compose"
+volumes = cfg.get("volumes") or {}
+services = cfg.get("services") or {}
+
+# ---- Detect leaky volumes -------------------------------------------------
+# A volume leaks past COMPOSE_PROJECT_NAME when it is external, or carries a
+# fixed top-level name (resolved name != the auto-namespaced <project>_<key>).
+forked = []          # (volkey, reason, pinned_name)
+shared_leaky = []    # (volkey, reason) — leaky but caller marked shareable
+for key, vol in volumes.items():
+    vol = vol or {}
+    external = bool(vol.get("external"))
+    resolved = vol.get("name", "")
+    auto = f"{project}_{key}"
+    if external:
+        reason = "external:true"
+    elif resolved and resolved != auto:
+        reason = f"fixed name '{resolved}'"
+    else:
+        continue  # already namespaced by project name — safe
+    if key in share:
+        shared_leaky.append((key, reason))
+        continue
+    pinned = f"{project}_wt_{wt_name}_{key}"
+    forked.append((key, reason, pinned))
+
+# ---- Detect services publishing fixed host ports --------------------------
+stripped = []  # service names whose host port publishing we drop
+for svc, spec in services.items():
+    spec = spec or {}
+    ports = spec.get("ports") or []
+    if any((p or {}).get("published") for p in ports):
+        stripped.append(svc)
+
+# ---- Emit the override YAML (hand-rolled; no yaml dependency) --------------
+def y(s):
+    # Quote scalars that YAML could misread; safe for our name/key charset.
+    return json.dumps(s)
+
+lines = []
+lines.append("# GENERATED by gen-compose-worktree-override.sh — do not edit by hand.")
+lines.append("# Out-of-tree compose override that neutralizes the isolation leaks")
+lines.append("# COMPOSE_PROJECT_NAME does not cover (external / fixed-name volumes,")
+lines.append("# fixed host port bindings). Wire in without touching tracked files via:")
+lines.append(f"#   COMPOSE_FILE={base}:{out_abs}   (in the worktree's .env)")
+lines.append(f"# Source project: {project}   Worktree: {wt_name}")
+if shared_leaky:
+    for key, reason in shared_leaky:
+        lines.append(f"# KEPT SHARED (caller --share-volume): {key} ({reason})")
+lines.append("")
+
+if stripped:
+    lines.append("services:")
+    for svc in sorted(stripped):
+        lines.append(f"  {y(svc)}:")
+        # `!override []` REPLACES the base list (plain [] would append/merge).
+        lines.append("    ports: !override []")
+    lines.append("")
+
+if forked:
+    lines.append("volumes:")
+    for key, _reason, pinned in forked:
+        lines.append(f"  {y(key)}:")
+        lines.append(f"    name: {y(pinned)}")
+        lines.append("    external: false")
+    lines.append("")
+
+if not stripped and not forked:
+    lines.append("# No isolation leaks detected: every volume is already namespaced by")
+    lines.append("# COMPOSE_PROJECT_NAME and no service publishes a fixed host port.")
+    lines.append("# This override is a valid no-op; wiring it in is harmless.")
+    lines.append("")
+
+with open(out, "w") as f:
+    f.write("\n".join(lines).rstrip("\n") + "\n")
+
+# ---- Report to stderr (human + agent readable) ----------------------------
+def err(msg):
+    print(msg, file=sys.stderr)
+
+err(f"gen-compose-worktree-override: wrote {out}")
+err(f"  base compose file: {base}   project: {project}")
+if forked:
+    err("  forked volumes (were shared → now worktree-namespaced, non-external):")
+    for key, reason, pinned in forked:
+        err(f"    - {key} ({reason}) -> {pinned}")
+else:
+    err("  forked volumes: none")
+if shared_leaky:
+    err("  KEPT SHARED (caller --share-volume) — verify these are read-only caches:")
+    for key, reason in shared_leaky:
+        err(f"    - {key} ({reason})")
+if stripped:
+    err(f"  stripped host-port publishing from: {', '.join(sorted(stripped))}")
+else:
+    err("  stripped host-port publishing from: none")
+err(f"  wire in: COMPOSE_FILE={base}:{out_abs}  (append to the worktree's copied .env)")
+
+# stdout = override path ONLY (safe for $(...) capture).
+print(out_abs)
+PY