@gotgenes/pi-autoformat 0.1.0 → 4.0.3

package/docs/plans/0004-shell-driven-mutation-coverage.md

@@ -0,0 +1,296 @@
+---
+issue: 4
+issue_title: "Post-v1: investigate shell-driven mutation coverage"
+---
+
+# Plan: Shell-Driven Mutation Coverage (Issue #4)
+
+## Problem Statement
+
+The v1 extension only tracks files mutated through Pi's built-in `write` and `edit` tools.
+Files modified by `bash` invocations — including codegen, codemods, `sed -i`, `mv`, downloaded scaffolds, and project-specific scripts — are invisible to the touched-files queue and therefore are not formatted.
+
+The original Issue #4 describes this as "one of the biggest remaining coverage gaps" for users who rely on shell commands that modify files.
+
+## Goals
+
+- Detect a useful subset of file mutations performed by shell commands.
+- Funnel any detected files through the existing prompt-end batching pipeline, reusing the formatter resolution and reporting paths.
+- Keep behavior explicit, opt-in, and predictable — no repository-wide scans.
+- Preserve the v1 safety properties: prompt-end timing remains the default, formatter failures stay non-blocking.
+
+## Non-Goals
+
+- Tracking arbitrary side effects of complex shell pipelines.
+- Filesystem watchers or whole-repo rescans after each `bash` call.
+- Running formatters inside the shell tool's lifetime (still prompt-end).
+- Strict-mode failure semantics (covered by Issue #6).
+
+## Background
+
+Relevant existing pieces:
+
+- `src/touched-files-queue.ts` — set-based dedupe of touched paths, currently hard-coded to the `write`/`edit` mutation tools.
+- `src/extension.ts` — registers the `tool_result` handler that feeds the queue and triggers the prompt-end flush via `agent_end`.
+- `src/prompt-autoformatter.ts` and `src/formatter-executor.ts` — already agnostic to where touched files came from.
+
+This means the extension architecture is mostly compatible with new mutation sources; the work is mostly in *detection* and *configuration*.
+
+## Design Overview
+
+We will add an opt-in shell mutation detector that participates in the same `tool_result` event flow used today.
+
+The detector is structured as a chain of strategies, each emitting candidate paths from a shell tool result.
+Strategies are intentionally narrow and explicit so users can reason about which commands are covered.
+
+### Strategy 1: Argument parsing for known mutating commands (default on)
+
+For a small whitelist of commands with well-known mutation flags, parse the shell input string and extract target file arguments.
+Initial whitelist:
+
+- `sed -i …`
+- `mv … <dest>` (single-target form only)
+- `cp … <dest>` (single-target form only)
+- `touch <files…>`
+- `> <file>` and `>> <file>` redirections at the top of a simple command
+- `tee <file>` / `tee -a <file>`
+
+Rules:
+
+- Only act when the parser recognizes the *whole* command shape; bail on pipelines, command substitutions, or unknown flags.
+- Resolve paths relative to `ctx.cwd`.
+- Drop paths that do not exist post-execution (the file may have been deleted by `mv` away, etc.).
+- Out-of-scope filtering is handled centrally by the queue (see *Format Scope* below).
+
+This covers the common deterministic cases without a generic shell parser.
+
+### Strategy 2: Pre/post directory snapshot for explicit globs (opt-in)
+
+Per-project config can declare *snapshot scopes* — globs whose mtimes are sampled before and after each shell tool call.
+Files whose mtime advanced are treated as touched.
+
+```jsonc
+{
+  "shellMutationDetection": {
+    "enabled": true,
+    "snapshotGlobs": ["src/**/*.ts", "docs/**/*.md"]
+  }
+}
+```
+
+Rules:
+
+- Snapshot only matched paths, not the whole repo.
+- Cap the number of snapshotted entries (e.g., 5,000) and warn on overflow.
+- Skip strategy entirely when no globs are configured, even if `enabled`.
+- Use `node:fs` `stat` only; never read file contents during snapshotting.
+- Skip directories in `.gitignore` and `node_modules` by default.
+
+This is the explicit, low-noise alternative to whole-repo heuristics called out in the issue's constraints.
+
+### Strategy 3: User-declared shell wrappers (opt-in)
+
+Allow users to configure shell command prefixes that are known to print the files they touched on stdout, one per line:
+
+```jsonc
+{
+  "shellMutationDetection": {
+    "wrappers": [
+      { "prefix": "pnpm codegen", "outputFormat": "lines" }
+    ]
+  }
+}
+```
+
+When a `bash` tool result matches a configured prefix, parse stdout for paths and enqueue them.
+This gives users a precise escape hatch without us having to model every codegen tool.
+
+## Format Scope (Out-of-CWD Handling)
+
+The v1 `TouchedFilesQueue` normalizes paths but does **not** filter out-of-cwd targets.
+Tightening this is necessary for the shell strategies (arg parsing and wrappers can produce arbitrary paths) and is also a latent v1 gap worth closing uniformly.
+
+### Default boundary: repo root, fall back to cwd
+
+At session start, resolve the format scope once:
+
+1. Run `git rev-parse --show-toplevel` from `ctx.cwd`.
+2. If it succeeds, use that path as the scope root.
+3. If it fails (not a Git repo, Git missing), fall back to `ctx.cwd`.
+
+This solves the monorepo case where Pi is launched inside a subpackage but the agent legitimately edits sibling packages, while staying conservative in non-Git contexts.
+Git is used only as a *boundary discovery* mechanism here — a much weaker coupling than the deferred `git status` detection strategy.
+
+### Configuration
+
+```jsonc
+{
+  "formatScope": "repoRoot"   // "repoRoot" | "cwd" | string[]
+}
+```
+
+- `"repoRoot"` (default): repo-root with cwd fallback, as above.
+- `"cwd"`: strict cwd subtree.
+- `string[]`: explicit allowlist of roots, each resolved relative to `ctx.cwd` at load time.
+
+### Identification rules
+
+For every candidate path, regardless of mutation source:
+
+1. Resolve to absolute via `path.resolve(cwd, candidate)`.
+2. `fs.realpath` both the candidate and each scope root, when the candidate exists.
+   Skip realpath if the candidate does not exist (e.g., a deleted target after `mv`); fall back to the normalized absolute form.
+3. Compute `path.relative(scopeRoot, resolvedCandidate)`.
+   In-scope iff the result is non-empty, does not start with `..`, and is not absolute.
+4. Use case-insensitive comparison on `darwin` and `win32`; case-sensitive elsewhere.
+5. If multiple scope roots are configured (the `string[]` form), the candidate is in-scope if it falls under any of them.
+
+Realpath on both sides is what makes this correct in the presence of symlinks:
+
+- A `pnpm` workspace dep symlinked into `node_modules` resolves *out* of the scope root and is correctly filtered.
+- A `vendor/lib` symlink pointing to an absolute path that realpaths *into* a configured scope root is correctly included.
+
+### Out-of-scope handling
+
+Drop silently from the queue.
+Out-of-scope paths are common and benign (`mv` to `/tmp/`, scratch edits in `~/`), so user-visible warnings would be noise.
+Optionally emit a debug-level log entry; do not surface in the prompt-end summary.
+
+### Applied uniformly
+
+The scope check runs in `TouchedFilesQueue` itself, after path normalization.
+All mutation sources — `write`, `edit`, shell argument parsing, wrappers, snapshot tracker — funnel through the same filter.
+Each strategy's rules can stop restating "drop paths outside cwd" since the queue enforces it centrally.
+
+### Migration note
+
+This tightens behavior for the existing `write`/`edit` paths: previously they would format any path the agent supplied.
+The new default of `repoRoot` (with cwd fallback) is almost certainly the behavior users already expect, but it is technically a change.
+Call it out in the changelog.
+Users who relied on the old behavior can configure `formatScope` to a broader allowlist; we deliberately do not provide a "no scope check" escape hatch.
+
+## Shell Detection Configuration
+
+New top-level config block under the existing extension-owned config files:
+
+```jsonc
+{
+  "shellMutationDetection": {
+    "enabled": false,
+    "argumentParsing": true,
+    "snapshotGlobs": [],
+    "wrappers": []
+  }
+}
+```
+
+Precedence: project overrides global (existing behavior).
+
+Defaults are intentionally conservative — feature is fully opt-in.
+Once a user enables it, `argumentParsing` defaults to true because it has a tight, auditable surface.
+
+Aligned updates required (per AGENTS.md):
+
+- `schemas/pi-autoformat.schema.json`
+- `docs/configuration.md`
+- `README.md`
+- TypeScript config loader (`src/config-loader.ts`, `src/formatter-config.ts`)
+
+## Code Changes
+
+1. **Config**
+   - Extend `AutoformatConfig` with `shellMutationDetection`.
+   - Validate types and unknown keys in `config-loader.ts`.
+
+2. **New module: `src/shell-mutation-detector.ts`**
+   - Pure functions: `parseKnownCommand(input)`, `matchWrapper(input, output, wrappers)`.
+   - Class `SnapshotTracker` for strategy 2, with `before()` / `after()`.
+   - No I/O at module load; injectable `fs`/`glob` for tests.
+
+3. **Touched-files queue**
+   - Generalize `MUTATION_TOOLS` into a registry of mutation-source handlers.
+   - Add a `bash` handler that delegates to the detector.
+   - Keep dedupe and `cwd`-relative normalization centralized.
+
+4. **Extension wiring**
+   - `tool_result` handler: if the tool is `bash` and detection is enabled, run argument parsing and wrapper matching against the result payload.
+   - Wrap the shell tool with `before/after` snapshot calls when `snapshotGlobs` is non-empty.
+     This requires a `tool_start` hook (or equivalent) — confirm the Pi extension API exposes one; if not, defer strategy 2 to a follow-up.
+
+5. **Reporting**
+   - No new reporting surface.
+     Files surface through the existing prompt-end summary path.
+
+## Testing
+
+Per AGENTS.md, add focused tests:
+
+- `shell-mutation-detector` argument parsing
+  - `sed -i 's/a/b/' foo.txt` → `["foo.txt"]`
+  - `sed -i.bak …` → `["foo.txt"]` (and ignores the `.bak`)
+  - `mv a.txt b.txt` → `["b.txt"]`
+  - pipelines / command substitutions → `[]`
+  - paths outside the format scope → `[]` (covered by the queue's scope check)
+- snapshot tracker
+  - mtime advances → reported
+  - mtime unchanged → ignored
+  - cap exceeded → warning emitted, partial result returned
+- wrapper matching
+  - prefix match with line output → list of files
+  - prefix mismatch → empty
+- queue integration
+  - `bash` tool result feeds into prompt-end flush alongside `write`/`edit`
+  - dedupe across sources works
+- format scope
+  - `repoRoot`: in-repo paths kept, out-of-repo paths dropped
+  - `repoRoot` with no Git: falls back to cwd subtree
+  - `cwd`: sibling-package paths dropped
+  - `string[]`: candidate matches any configured root
+  - symlinked workspace dep (realpaths outside scope) is dropped
+  - symlink whose realpath lands inside scope is kept
+  - case-insensitive match honored on darwin/win32
+- config loader
+  - defaults are off
+  - project override of `snapshotGlobs` replaces, does not merge with global (consistent with current array-override behavior — confirm and document)
+
+## Rollout
+
+1. Land `formatScope` config + uniform scope filtering in the queue (independent of detection — closes the v1 gap on its own).
+2. Land config plumbing + schema/docs updates for shell detection with detection disabled.
+3. Land argument-parsing strategy behind the new config flag.
+4. Land wrapper strategy.
+5. Land snapshot strategy if a `tool_start` (or pre-tool) hook exists; else open a follow-up issue.
+6. Update `docs/plans/0001-initial-implementation-plan.md` and the README to describe the new opt-in coverage, the format scope behavior, and the explicit constraints.
+
+## Open Questions
+
+- Does the Pi extension API expose a pre-tool hook usable for snapshotting? If not, strategy 2 is deferred.
+- Should wrappers support a JSON output format in addition to line format? Likely yes, but not in the first cut.
+
+## Explicitly Deferred: `git status --porcelain` Detection
+
+Using `git status --porcelain` (with a pre-call snapshot diff) was considered as a fourth strategy.
+It offers near-complete coverage with no per-command modeling, honors `.gitignore` for free, and is fast.
+
+It is **not** included at this stage because:
+
+- It is implicit repo-wide behavior, which Issue #4 explicitly warns against.
+- It produces false positives from concurrent activity (IDE saves, watchers, dev servers writing into tracked paths).
+- It interacts awkwardly with pre-existing dirty working trees and requires careful snapshot/diff logic to subtract unrelated in-progress edits.
+- Untracked files are ambiguous — sweeping them in catches scratch files, logs, and downloaded artifacts.
+- It silently does nothing in non-Git directories, which Pi runs in more often than expected.
+- It loses per-command attribution, hurting debuggability and foreclosing future per-command policy.
+
+The explicit strategies above match the issue's stated philosophy that "explicit, low-noise designs are preferable to implicit heuristics." We can revisit `git status` later if real-world usage shows meaningful gaps the three explicit strategies cannot close, ideally as a scoped, opt-in tertiary strategy rather than a default.
+
+## Checkpoints / Commits
+
+Following Conventional Commits:
+
+- `feat: add formatScope config with repo-root default and cwd fallback`
+- `feat(config): add shellMutationDetection schema and loader support`
+- `feat: parse known mutating shell commands into touched files`
+- `feat: support user-declared shell wrappers for touched-file output`
+- `feat: snapshot configured globs around bash tool calls` *(conditional)*
+- `docs: document shell mutation coverage and its constraints`
+- `test: cover shell mutation detector strategies and queue integration`

package/docs/plans/0010-acceptance-test-coverage.md

@@ -0,0 +1,240 @@
+---
+issue: 10
+issue_title: "Expand acceptance test coverage with end-to-end pi CLI scenarios"
+---
+
+# Plan: Expand acceptance test coverage (Issue #10)
+
+## Problem Statement
+
+`test/acceptance.test.ts` spawns the real `pi` CLI in `--mode rpc`, loads the extension, and verifies a single `get_state` round-trip.
+That smoke test catches load-time regressions (entrypoint shape, module resolution, `session_start` failures) without burning LLM credits, but it never drives a `tool_result` event end to end.
+Issue #10 asks us to expand acceptance coverage so we also catch payload-shape drift, custom-tool dispatch regressions, EventBus channel regressions, and prompt-end batching regressions against the real Pi runtime — while keeping `pnpm test` deterministic, offline, and skippable for contributors without `pi` installed.
+
+## Goals
+
+- Add deterministic, non-LLM acceptance tests that exercise the formatter pipeline through the real `pi` CLI (RPC mode), covering at minimum:
+  - the `bash` mutation path (snapshot tracker + shell mutation detector) driven by `{"type": "bash", ...}` RPC commands;
+  - the `pi.events`-based `autoformat:touched` channel via a small companion extension loaded alongside ours;
+  - the `customMutationTools` declarative path via a companion extension that registers a synthetic mutation tool and triggers it through an extension command (`/cmd`).
+- Reuse the existing skip-when-`pi`-is-absent pattern so `pnpm test` stays green for contributors without Pi installed.
+- Keep each acceptance test isolated to its own `cwd` and config, with no shared state between tests.
+- Document an opt-in, env-gated path (`PI_AUTOFORMAT_LLM_TESTS=1`) for occasional LLM-backed scenarios — design only, no LLM calls in default CI.
+- Resolve the `pi` binary from the locally-installed `@mariozechner/pi-coding-agent` devDependency rather than the global `PATH`, so the acceptance suite runs in CI under the existing `pnpm install --frozen-lockfile` step with no workflow changes.
+
+## Non-Goals
+
+- Running LLM-backed scenarios on every `pnpm test` invocation.
+  These remain opt-in behind `PI_AUTOFORMAT_LLM_TESTS=1`; default CI must not require API keys.
+- Adding new product features.
+  This is a test-coverage change; the only production code touched would be small fixes to anything the new tests expose.
+- Replacing the existing unit / integration tests.
+  Acceptance tests are additive; they pin behavior at the Pi-runtime boundary, not inside our modules.
+- Building a generic Pi-extension test harness.
+  The fixtures live under `test/fixtures/` and exist solely to drive the autoformatter pipeline.
+- Hardening Pi's RPC protocol or filing upstream requests beyond what this plan needs.
+
+## Background
+
+Relevant existing surface:
+
+- `test/acceptance.test.ts` — current smoke test.
+  Spawns `pi --mode rpc --no-tools --no-extensions --no-session -e <EXTENSION_PATH>`, writes JSON commands to stdin, parses JSON-per-line responses from stdout.
+  Today it relies on `pi` being on `PATH` and skips when `spawnSync("pi", ["--help"]).status !== 0`; the new harness will resolve `pi` from `node_modules/.bin/pi` so the test runs whenever `pnpm install` has been done.
+- `src/extension.ts` — `createAutoformatExtension` wires three real touched-file sources: built-in `write`/`edit` `tool_result` events, `customMutationTools` declared in config, and `pi.events.on(channel, …)` for the `autoformat:touched` (configurable) channel.
+- `src/shell-mutation-detector.ts` — drives `bash` snapshot tracking (`SnapshotTracker`) plus argument parsing and wrapper matching for known mutation commands. The `bash` RPC command sends a real `tool_call` + `tool_result` pair through Pi.
+- `src/custom-mutation-tools.ts` — `parseTouchedPayload` and `createCustomToolHandlers` accept either a `{ touched: string[] }` payload (event-bus path) or extract paths from configured custom tools.
+- `node_modules/@mariozechner/pi-coding-agent/docs/rpc.md` — confirms RPC supports `prompt`, `bash`, `get_state`, and that `prompt` accepts extension commands (`/mycommand`) which execute immediately even without an LLM provider configured.
+- `node_modules/@mariozechner/pi-coding-agent/docs/extensions.md` — confirms `pi.registerCommand(name, { handler })` lets a companion extension expose a slash command we can trigger over RPC.
+
+Implications:
+
+- Pi's RPC mode does not document a way to inject synthetic `tool_result` events directly.
+  We therefore drive real events via two channels Pi already provides: the `bash` RPC command (real `bash` `tool_result`) and `prompt` with a slash command (lets a companion extension synthesize side effects).
+- A companion extension is the cleanest way to exercise both `customMutationTools` and the `autoformat:touched` event-bus channel without an LLM.
+- Multiple `-e` flags are supported (per `extensions.md`), so we can load `src/extension.ts` and a fixture extension in the same RPC session.
+
+## Design Overview
+
+### Test fixtures
+
+Place small companion extensions under `test/fixtures/` so the production package never imports them:
+
+```text
+test/fixtures/
+  event-bus-emitter.ts        # registers /emit-touched, calls pi.events.emit("autoformat:touched", { touched })
+  custom-tool-emitter.ts      # registers a synthetic mutation tool + /trigger-custom-tool slash command
+  formatter-recorder.sh       # POSIX shell script used as the formatter "command" so we can assert invocation
+```
+
+`formatter-recorder.sh` writes its argv plus `cwd` to a file the test can read after the flush.
+Using a real on-disk recorder keeps the test independent of mocking — it asserts what Pi actually invoked.
+
+### Shared RPC harness
+
+Extract the `runRpcSession(...)` helper currently inline in `test/acceptance.test.ts` into `test/helpers/rpc.ts` so all acceptance tests share one implementation.
+The helper accepts an optional `extraExtensions: string[]` so each test can load its companion fixture alongside `src/extension.ts`.
+
+```typescript
+// test/helpers/rpc.ts
+export type RpcResponse = {
+  id?: string;
+  type: string;
+  command?: string;
+  success?: boolean;
+  data?: unknown;
+};
+
+export type RpcEvent = { type: string; [key: string]: unknown };
+
+export async function runRpcSession(options: {
+  cwd: string;
+  commands: object[];
+  extraExtensions?: string[];
+  timeoutMs?: number;
+  env?: NodeJS.ProcessEnv;
+}): Promise<{
+  responses: RpcResponse[];
+  events: RpcEvent[];
+  stderr: string;
+  exitCode: number | null;
+}>;
+```
+
+### Per-test scaffolding
+
+Each acceptance test creates its own temp `cwd`, writes:
+
+- `.pi/extensions/pi-autoformat/config.json` configuring formatters whose `command` points at `formatter-recorder.sh` (or a small Node runner script copied next to the temp dir);
+- one or more files matching the chain extensions, so the formatter actually has something to run on.
+
+After the RPC session closes, the test reads the recorder log and asserts:
+
+- which formatter command ran;
+- which absolute paths were passed in argv;
+- per-test invariants (e.g. the snapshot tracker only emitted files that actually changed during the bash command).
+
+### Scenarios
+
+1. `bash` shell-mutation acceptance.
+   - `commands: [{type: "bash", command: "node -e 'fs.writeFileSync(\"out.ts\", ...)'"}]` followed by an explicit `agent_end` trigger (we already flush on `agent_end`; for the bash-only scenario we trigger the prompt-end flush by sending a `prompt` with a no-op slash command provided by a tiny "flush trigger" fixture, or by closing stdin and asserting the `session_shutdown` flush — pick whichever is more deterministic during TDD).
+   - Asserts the recorder ran on `out.ts` exactly once.
+
+2. `customMutationTools` acceptance.
+   - Loads `custom-tool-emitter.ts` via `-e`. The companion registers a tool whose name is also listed in `customMutationTools` config; the slash command `/trigger-custom-tool` calls `pi.sendMessage()`-style hooks that produce a real `tool_result` for that tool with a `{touched: [...]}` payload.
+   - If we cannot trigger a real registered tool without an LLM, fall back to a slash command that emits the synthetic `tool_result` via a documented hook; if no such hook exists, drop this scenario into Open Questions and rely on the EventBus path for v1.
+   - Asserts the recorder ran on the declared file.
+
+3. `autoformat:touched` EventBus acceptance.
+   - Loads `event-bus-emitter.ts` via `-e`. Slash command `/emit-touched <path>` calls `pi.events.emit("autoformat:touched", { touched: [absolutePath] })`.
+   - Test sends `{type: "prompt", message: "/emit-touched out.ts"}`, then triggers a flush as in (1).
+   - Asserts the recorder ran on `out.ts`.
+
+4. LLM-gated scenario (design only; not implemented in this plan unless trivial).
+   - Skipped unless `process.env.PI_AUTOFORMAT_LLM_TESTS === "1"` and a provider env var is present.
+   - Sends a real `prompt` and asserts the recorder ran on the file the agent edited.
+   - Documented in `docs/testing.md` (new), not enabled in CI.
+
+### Resolving the `pi` binary
+
+`@mariozechner/pi-coding-agent` is already a devDependency and ships a `pi` bin, so `pnpm install` produces a working `node_modules/.bin/pi`.
+The new harness resolves that path explicitly instead of relying on the global `PATH`:
+
+```typescript
+// test/helpers/rpc.ts
+import { existsSync } from "node:fs";
+import { resolve } from "node:path";
+
+export const PI_BIN = resolve("node_modules/.bin/pi");
+export const piAvailable = existsSync(PI_BIN);
+```
+
+Benefits:
+
+- CI runs the acceptance suite with no workflow changes — `pnpm install --frozen-lockfile` already provides `pi`.
+- The Pi version is pinned by `pnpm-lock.yaml`, so CI and local runs use the same binary.
+- Contributors who ran `pnpm install` immediately get the acceptance suite; no separate global install.
+
+### Skip semantics
+
+`describeIfPi` becomes a safety net rather than the default contributor experience: it skips only when `node_modules/.bin/pi` is missing (e.g. someone forgot `pnpm install`).
+LLM-gated scenarios add a second guard on `PI_AUTOFORMAT_LLM_TESTS` and the relevant API key.
+
+## Module-Level Changes
+
+- `test/helpers/rpc.ts` — new. Extracted RPC harness. Adds `extraExtensions`, `env`, and event/response separation.
+- `test/acceptance.test.ts` — updated to import from `test/helpers/rpc.ts`. Behavior unchanged.
+- `test/acceptance-bash-mutation.test.ts` — new. Scenario (1).
+- `test/acceptance-event-bus.test.ts` — new. Scenario (3).
+- `test/acceptance-custom-tool.test.ts` — new. Scenario (2). May be deferred if the no-LLM trigger turns out infeasible (see Open Questions).
+- `test/fixtures/event-bus-emitter.ts` — new companion extension.
+- `test/fixtures/custom-tool-emitter.ts` — new companion extension.
+- `test/fixtures/formatter-recorder.sh` — new helper script (executable, POSIX `sh`).
+- `docs/testing.md` — new. Documents acceptance-test layout, the `node_modules/.bin/pi` resolution behavior, skip semantics, and the env-gated `PI_AUTOFORMAT_LLM_TESTS` design.
+- `README.md` — small "Testing" pointer to `docs/testing.md`.
+
+No `.github/workflows/*.yml` change is needed: the existing `pnpm install --frozen-lockfile` step already provides `pi` via the devDependency.
+
+No production source under `src/` is expected to change.
+If the new tests expose a real bug, that bug is fixed in its own commit on the same branch.
+
+## TDD Order
+
+1. **Refactor the RPC harness (red → green).**
+   Move `runRpcSession` into `test/helpers/rpc.ts`; add `extraExtensions` and an `events` array in the result.
+   Resolve `PI_BIN` from `node_modules/.bin/pi` and key `piAvailable` off `existsSync(PI_BIN)` instead of `spawnSync("pi", ["--help"])`.
+   Update `test/acceptance.test.ts` to use the new harness; the existing test must still pass.
+   Commit: `test: extract shared rpc harness and resolve pi from node_modules`.
+
+2. **Bash mutation acceptance (red → green).**
+   Add `test/fixtures/formatter-recorder.sh`. Add `test/acceptance-bash-mutation.test.ts` that writes a project config pointing at the recorder, sends a `bash` RPC command that creates `out.ts`, triggers a flush, and asserts the recorder log.
+   Commit: `test: add acceptance coverage for bash-driven mutation flush`.
+
+3. **EventBus channel acceptance (red → green).**
+   Add `test/fixtures/event-bus-emitter.ts` and `test/acceptance-event-bus.test.ts`. Drive `/emit-touched` via `prompt`, trigger a flush, assert.
+   Commit: `test: add acceptance coverage for autoformat:touched event bus`.
+
+4. **Custom-tool acceptance (red → green) — conditional.**
+   Add `test/fixtures/custom-tool-emitter.ts` and `test/acceptance-custom-tool.test.ts`. If the no-LLM trigger path proves infeasible during TDD, capture the finding in Open Questions and skip this cycle.
+   Commit: `test: add acceptance coverage for customMutationTools dispatch`.
+
+5. **Documentation (green → docs).**
+   Add `docs/testing.md` describing the `node_modules/.bin/pi` resolution, skip semantics, and the env-gated LLM scenario design.
+   Update `README.md` to point at it.
+   Commit: `docs: document acceptance-test layout and pi binary resolution`.
+
+## Risks and Mitigations
+
+- **RPC has no synthetic-tool-result injection.**
+  Mitigation: drive real events via `bash` RPC and slash-command-emitted EventBus events.
+  The custom-tool scenario degrades gracefully: if no non-LLM trigger exists, mark it deferred and rely on the EventBus path until an LLM-gated test fills the gap.
+
+- **Spawning `pi` is slow and platform-sensitive.**
+  Mitigation: keep per-test timeout generous (10 s default, configurable per test), parallelize sparingly (Vitest runs files in parallel — these tests get their own files for `cwd` isolation), and continue skipping when `pi` is absent.
+
+- **Formatter recorder script is not portable to Windows.**
+  Mitigation: use a tiny Node script (`formatter-recorder.mjs`) instead of `.sh` if Windows support matters.
+  Default plan picks the script flavor that matches the maintainer's CI; fallback is documented in `docs/testing.md`.
+
+- **Companion extensions drift from Pi's API.**
+  Mitigation: keep them tiny (≤ 30 lines each), import from the same `@mariozechner/pi-coding-agent` types the production extension uses, and exercise them in CI so any drift fails fast.
+
+- **Snapshot tracker false negatives.**
+  The `bash` scenario depends on the snapshot tracker's globs matching the test file.
+  Mitigation: explicitly configure `shellMutationDetection.snapshotGlobs` in the project config the test writes, so the test pins the configured contract rather than the default.
+
+## Open Questions
+
+- Should `formatter-recorder` be a POSIX shell script or a Node script?
+  Resolved during execution: chose `formatter-recorder.mjs` (Node) for portability.
+
+## Execution Notes
+
+- Step 2 (bash mutation acceptance) and step 4 (`customMutationTools` acceptance) were **deferred** during execution.
+  Empirical probing (and Pi's `docs/rpc.md`) confirmed that the RPC `bash` command does not emit `tool_call` / `tool_result` events; it only stores a `BashExecutionMessage` for the next prompt's LLM context.
+  Likewise, slash commands run extension code directly without going through tool dispatch, so a fixture extension cannot synthesize a `tool_result` event for a registered custom tool.
+  Both scenarios therefore require a real LLM-driven tool invocation and have been moved into the future LLM-gated suite documented in `docs/testing.md`.
+- The plan's example payload `{ touched: string[] }` for the EventBus channel was incorrect.
+  The real contract handled by `parseTouchedPayload` is `{ path: string }` or `{ paths: string[] }`; the fixture and test now use `{ paths }`.
+- macOS resolves `/var` to `/private/var` via realpath.
+  The acceptance test calls `realpathSync` on its temp `cwd` so assertions on the recorder's `process.cwd()` match what Pi spawns the formatter with.