pan-wizard 3.36.0 → 3.37.0

package/docs/specs/investigations/digests/2026-10-10.md

@@ -0,0 +1,61 @@
+# Investigation — 2026-10-10
+
+Source: the `mattpocock/skills` repository (`github.com/mattpocock/skills`, MIT, `main` at `49dd158`, `2026-10-09`), read from the GitHub API; each skill's frontmatter, plus the full `code-review` and `diagnosing-bugs` bodies · no local copy (public repository, cited by commit) · ADD 0 · ENHANCE 2 · HAVE 7 · WATCH 1 · SKIP 4
+
+## Recommendations
+1. **Debugger: a failing command before any hypothesis**: ENHANCE, P2, S. `pan-debugger` ranks hypotheses but treats reproduction as an end check. Make "run a command that goes red on the reported symptom, then shrink it" the first step, record it in the session file, and hand it to the fix as its check. → [write-up](../features/debugger-red-loop-first.md)
+2. **Reviewer: a fixed code-smell baseline**: ENHANCE, P3, S. Twelve named design smells, always INFO, phrased as "possible …", overridden by the project's own standards. → [write-up](../features/reviewer-smell-baseline.md)
+
+The repository is a skill library for one engineer's workflow, wired to an issue tracker. Most of it maps onto things PAN does through `.planning/` instead. The two items worth taking are disciplines inside skills, not new commands.
+
+## Triage
+
+| # | Item | Area | Relevance | Verdict | Ledger |
+|---|---|---|---|---|---|
+| 1 | `diagnosing-bugs`: a red-capable, minimised reproduction before any hypothesis | agents | HIGH | ENHANCE → rec. 1 | — |
+| 2 | `code-review`: Standards and Spec reviewed separately, in parallel, never merged | agents | MED | HAVE | — |
+| 3 | `code-review`: a fixed smell baseline as labelled heuristics | agents | MED | ENHANCE → rec. 2 | — |
+| 4 | `tdd`: red-green-refactor, integration tests | orchestration/workflows | LOW | HAVE | — |
+| 5 | `retro`: a retrospective on a coding session | learnings/optimize | LOW | HAVE | — |
+| 6 | `handoff`: compact the conversation into a document for another agent | planning state | LOW | HAVE | MI-036 |
+| 7 | `to-spec`, `to-tickets`, `wayfinder`, `implement`: plans as tickets with blocking edges | planning state | MED | HAVE | MI-018 |
+| 8 | `grilling`, `grill-me`, `grill-with-docs`: a relentless interview on a plan | orchestration/workflows | LOW | HAVE | — |
+| 9 | `research`: primary-source research written to a file | agents | LOW | HAVE | — |
+| 10 | `triage`, `pr`, `setup-matt-pocock-skills`: issue-tracker state machines and PR bodies | other | LOW | SKIP: built on GitHub or GitLab issue trackers; PAN keeps its state in `.planning/` and depends on no tracker | — |
+| 11 | `git-guardrails-claude-code`, `setup-pre-commit`: hooks that block git commands, Husky pre-commit | security/supply-chain | LOW | SKIP: project tooling choices a user makes, not PAN's; PAN's own commit guard covers what PAN commits | — |
+| 12 | TypeScript and teaching skills (`setup-ts-deep-modules`, `migrate-to-shoehorn`, `scaffold-exercises`, `teach`) | other | NONE | SKIP: language- or course-specific | — |
+| 13 | Writing skills (`writing-for-agents`, `writing-beats`, `writing-shape`, `writing-fragments`) | docs/positioning | LOW | SKIP: authoring aids for people; PAN's prompt rules live in its own tests and doc lints | — |
+| 14 | Installing these skills puts project-owned skills where PAN's scope check treats every entry as PAN's (`.agents/`) | planning state | LOW | WATCH | — |
+
+## HAVE — already covered
+- **2, two-axis review.** `pan-reviewer` reviews against discovered project conventions and flags unrequested work (`agents/pan-reviewer.md:39-44`, `:72`). `pan-verifier` checks goal achievement backward from the phase's must-haves. Both are fresh-context agents spawned separately in exec-phase. SOURCE.
+- **4, TDD.** `pan-wizard-core/references/tdd.md` and TDD plans with `verify plan-structure`. SOURCE.
+- **5, retro.** `/pan:retro`. SOURCE.
+- **6, handoff.** `/pan:pause` writes `.continue-here.md`. `state record-session`, and the re-injection hook after compaction, restore position (MI-036). SOURCE.
+- **7, specs and tickets.** Roadmap phases, plans with `depends_on` and waves, and the army's campaign. They live in `.planning/`, not an issue tracker; MI-018 keeps that layout PAN's own. SOURCE.
+- **8, grilling.** `/pan:discuss-phase` and `/pan:assumptions` question the user before planning. SOURCE.
+- **9, research.** `pan-phase-researcher` and the project researchers, with primary sources and confidence levels. SOURCE.
+
+## WATCH
+- **14, project skills under `.agents/`.** PAN treats the whole `.agents/` tree as its own in three places:
+  - the scope check (`verify-scope.cjs` `PAN_OWNED_PREFIXES`);
+  - the reviewer's skip list (`agents/pan-reviewer.md:48`);
+  - the mapping and scan exclusions (`map-codebase.md`, `phase-tests.md`, the focus commands).
+
+  Codex and unified-skills installs write `.agents/skills/pan-*` and `.agents/pan-wizard-core/`. A project's own skills there would also be skipped. Excluding them from mapping and scans is arguably right, since skills are not source code. Only the scope check loses something: a phase that edits a project skill is never checked for unrequested work.
+
+  `.github/` was narrowed to `pan-*` entries on `2026-10-05`. The same narrowing here (`.agents/skills/pan-`, `.agents/pan-wizard-core/`) is cheap, but nothing shows the case happening. Where the skills installer writes per agent was not confirmed; the repository README does not say.
+
+  Trigger: a field project, or a user report, with project skills edited inside a phase.
+
+## Hand-offs to /reality-check
+None.
+
+## How far to trust the material
+- **Primary, but practitioner guidance.** This is a public repository read at a pinned commit. The skills are one engineer's workflow, not measured results: nothing in it reports an outcome comparison, so the two ENHANCE items rest on the discipline being sound, not on evidence that it changes outcomes.
+- **The reproduction gate is consistent with PAN's own evidence-over-assertion rules**, and checking it is cheap. The smell baseline is standard software-engineering vocabulary (Fowler); its value in an AI reviewer is untested.
+- **Popularity is secondary.** The star count the API reports (over 280,000) shows reach, not quality.
+
+## Notes
+- The material contains no text addressed to the agent.
+- The repository's own out-of-scope records (`.out-of-scope/`) include "subagent recursion" and "harness name collisions". Not read in full; noted as possible input for a later run.

package/docs/specs/investigations/features/debugger-red-loop-first.md

@@ -0,0 +1,55 @@
+# Debugger: a failing command before any hypothesis
+
+| | |
+|---|---|
+| Verdict | ENHANCE |
+| Priority | P2 (this release cycle) |
+| Size | S 2 points |
+| Area | agents |
+| Runtimes | all five (agent prompt only) |
+| First seen | `2026-10-10` — [digest](../digests/2026-10-10.md) |
+| Ledger | MI-105 |
+| Status | FILED MI-105 |
+
+## Source trail
+- `2026-10-10` — Matt Pocock's `diagnosing-bugs` skill puts a hard gate first. The agent may not theorise until it has run one command that goes red on the user's exact symptom. That command must give the same verdict every run, finish in seconds, and run without a human. It is then shrunk until every remaining part is needed for the failure. Only after that does the agent list 3–5 ranked hypotheses. (source: `github.com/mattpocock/skills`, `skills/engineering/diagnosing-bugs/SKILL.md` at `49dd158`, `2026-10-09`, MIT)
+
+## Why it matters for PAN
+`pan-debugger` is spawned by `/pan:debug`, by exec-phase's UAT diagnosis and by the native `pan-diagnose-issues` script. Its job is a root cause the executor can act on.
+
+A diagnosis built on a hypothesis that was never tested against a red command is the failure the skill names: the agent reads code, builds a story and "fixes" a neighbouring bug. PAN's verification pillar asks for evidence over assertion. A reproduction command the debugger has run is that evidence. It also hands the fix its regression test.
+
+## Current state (evidence)
+- **The hypothesis half is there.** Three or more independent hypotheses with priors, the top two investigated in parallel, and the tree recorded in the debug file (`agents/pan-debugger.md:131-139`). Generating several hypotheses before investigating is the anti-anchoring rule (`:76`). SOURCE.
+- **Reproduction is only an end-of-session check.** "Reproduce reliably?" sits among the criteria for calling the root cause found (`:175`), not as a step before theorising. No rule says the session file must record the command that reproduces the symptom. Nothing asks for the repro to be minimised. SOURCE.
+
+## Proposal
+Add a first step to the debugger's investigation, before the hypothesis tree:
+- **Build a reproduction command and run it.** The agent writes one command (a test, a CLI call with a fixture, a script) that fails on the symptom the user reported, and runs it. Command and output are recorded under a new `Reproduction` section of the debug session file, with secrets redacted.
+- **It must catch this bug.** The command must assert the reported symptom, not merely "runs without error". It must give the same verdict across runs; for a flaky bug, record the reproduction rate instead.
+- **Shrink it.** Cut inputs and steps one at a time while it stays red.
+- **When no command can be built,** say so and list what was tried. In `find_root_cause_only` mode return a checkpoint asking for the missing access or artifact, rather than theorising.
+- **The command becomes the fix's check.** In `find_and_fix` mode the fix is done only when the same command goes green. In diagnose mode the root-cause report names it, so the gap-closure plan can turn it into its test.
+
+## Implementation sketch
+| Layer | Change |
+|---|---|
+| Core module | none |
+| Dispatcher | none |
+| Installer / per-runtime | none |
+| Commands / agents / workflows | `agents/pan-debugger.md`: the new first step and its completion criterion; the `Reproduction` section in its File Structure. `pan-wizard-core/templates/debug.md`: the same section. `pan-wizard-core/workflows/diagnose-issues.md` and `commands/pan/debug.md`: say the returned diagnosis names the reproduction command. |
+| Hooks / MCP | none |
+| Tests | a prompt-shape test that the debugger's investigation step names the reproduction gate before the hypothesis tree, and that the template and the agent's File Structure carry the same section (the existing debug template/agent agreement tests are the pattern) |
+| Harness | optional tier-1 scenario: a seeded bug, `/pan:debug`, assert the session file records a command that fails on the seed and passes after the fix |
+| Docs | `docs/AGENTS.md` pan-debugger section |
+
+## Gate
+The prompt-shape test, shown failing before the change. If the owner wants a behavioural check, the optional harness scenario: the debug file records a command that is red on the seeded bug.
+
+## Effort & risk
+- Small and prompt-only.
+- A strict gate can stall on bugs that only reproduce in an environment the agent cannot reach. The checkpoint return keeps that honest instead of guessing.
+- Session-file readers that parse sections must tolerate the new one: check the `debug.md` parsers before adding it.
+
+## Open questions for the owner
+- Hard gate (no hypotheses without a red command) or strong default (allowed with a recorded reason)?

package/docs/specs/investigations/features/harness-cross-host-behaviour.md

@@ -0,0 +1,93 @@
+# Behavioural harness runs on every host CLI
+
+| | |
+|---|---|
+| Verdict | ENHANCE |
+| Priority | P2 (this release cycle) |
+| Size | L 10 points (all four non-Claude hosts); M 4 for the first host and the shape |
+| Area | harness/evals/testing |
+| Runtimes | Codex, Copilot CLI, Gemini CLI and OpenCode measured beside Claude Code; nothing shipped changes |
+| First seen | `2026-10-08` — [digest](../digests/2026-10-08.md) |
+| Ledger | MI-104 (related: MI-044, MI-071) |
+| Status | FILED MI-104 |
+
+## Source trail
+- `2026-10-08` — Hugging Face ran one set of small-model weights through four unmodified coding-agent harnesses and got very different pass rates on the same held-out tasks: 62.1% under Mini-SWE-Agent, 40.0 under Codex, 33.6 under OpenCode and 33.2 under Claude Code. Training through one harness transferred poorly to another: OpenCode-only RL scored 52.3 overall but 42 under Claude Code. (source: "The ultimate guide to multi-harness RL", Hugging Face with Liquid AI, published `2026-09-24`, updated `2026-10-01`, read via the VentureBeat AI Weekly of `2026-10-08`)
+
+## Why it matters for PAN
+PAN's product claim is one planning, execution and verification layer that behaves the same under five host tools. Its evidence for that is behavioural on one host only. The harness pillar (ADR-0047, "behavioural evals against deployed installs") checks behaviour through `claude -p`. On the other four hosts it checks that PAN installs and is discovered, and nothing more.
+
+The Hugging Face result is outside evidence that the host shapes outcomes, even with everything else held fixed. PAN's own record says the same from the other side. The host differences that mattered were all found by running the host live, never by reading its docs:
+- Codex cuts plugin skills at 8,000 bytes (MI-015, found by the M25 live probe).
+- The Codex gate wrote into the user's real `CODEX_HOME` (M31).
+- Gemini's hooks were registered under event names Gemini skips, from 3.4 until `2026-09-23`.
+
+Every one of those was an install or discovery fault. Nothing yet tells PAN whether a `/pan:exec-phase` chain completes under Codex or Copilot, or how much it costs there.
+
+The cost of not doing it: tuning decided on Claude-only measurements is assumed to hold on four hosts that were never measured. That covers the R21 body-split A/B, the O6 memory retirement and the chain timings. The Hugging Face transfer result is a warning against exactly that assumption.
+
+## Current state (evidence)
+- **Model steps drive Claude Code only.** `modelArgs()` builds a `claude` argv (`harness/src/model.cjs:46-55`), and `runModelStep()` spawns `claude` (`harness/src/model.cjs:63`). ADR-0047 D4 says the same in prose: "`model` runs `claude -p` in the workspace". SOURCE.
+- **Non-Claude coverage is install and discovery.**
+  - `live-gate-codex`, `live-gate-copilot`, `live-gate-gemini` and `live-gate-opencode` are tier 0. Their `cli` steps list plugins, MCP servers and agents.
+  - `copilot-agent-frontmatter` is the one paid non-Claude step: a single `copilot --agent … -p` turn that checks a frontmatter shape loads.
+
+  SOURCE (`harness/scenarios/*.json`).
+- **The chain scenarios are Claude-pinned.** `markdown-exec-phase-chain` installs `--claude`, requires `cli: claude` and prompts `/pan:exec-phase 1 --skip-tests`, budget `maxUsd: 8`. `native-exec-waves-chain` is Claude-only by nature (native workflows are a Claude Code feature, MI-002). SOURCE.
+- **Four of the five host CLIs are on this machine, each with a headless JSON mode.** PROBED `2026-10-08` (help output only, no model call):
+  - codex-cli 0.157.1: `codex exec --json`, plus `--dangerously-bypass-approvals-and-sandbox` and `--skip-git-repo-check`.
+  - GitHub Copilot CLI 1.0.91: `-p`, `--output-format json` (JSONL) and `--allow-all-tools`.
+  - Gemini CLI 0.61.0: `-p`, `-o json|stream-json` and `--approval-mode yolo`.
+  - OpenCode 1.18.32: `run --format json`.
+  - Antigravity (`agy`) is absent.
+- **Spend is priced and capped for Claude only.** `--max-budget-usd` is passed to `claude` (`model.cjs:50`), and the step cost comes from Claude's JSON. The other CLIs' usage fields are not read anywhere in the harness. SOURCE.
+- **What is missing:**
+  - a host-selectable model step;
+  - per-host cost accounting;
+  - a portable chain scenario run per host.
+
+## Proposal
+- **A `runtime` field on `model` steps.** It defaults to `claude`, so every existing scenario is unchanged. One adapter per host covers:
+  - the argv, including the flags an unattended run inside the scratch workspace needs;
+  - how the prompt is passed (stdin where the CLI accepts it, which avoids Windows `.cmd` quoting, as the Claude step already does);
+  - the output parser (final message, error flag, usage).
+- **Cost under `--max-usd` stays mandatory.**
+  - When the CLI reports dollars, use them.
+  - When it reports tokens, price them with PAN's own rate table (`cost.cjs`). MI-023 already prices the runtime defaults with the pricing page cited.
+  - A step whose CLI reports neither is refused, because the cap could not be enforced. ADR-0047 D2's rule stays as it is.
+- **One portable chain per host.** It uses the seed and assertions of `markdown-exec-phase-chain`, installed with that host's flag. The prompt is written in the host's command syntax, taken from the installer's own rewrite (`$pan-exec-phase` on Codex, `/pan-exec-phase` on OpenCode and Copilot; Gemini's form to confirm). A missing CLI skips with its reason, never green (D4's existing rule).
+- **Results per host.** The run report and the ledger signature carry the runtime and the CLI version, so a finding on one host is not merged with the same step on another.
+
+## Implementation sketch
+| Layer | Change |
+|---|---|
+| Core module (`pan-wizard-core/bin/lib/`) | none (the harness may `require` `cost.cjs` for its rate table) |
+| Dispatcher (`pan-tools.cjs`) | none |
+| Installer / per-runtime (`bin/install-lib.cjs`) | none; the chain prompt reuses the installer's command-syntax rewrite rather than a second table |
+| Commands / agents / workflows (markdown) | none |
+| Hooks / MCP | none |
+| Harness | `harness/src/model.cjs`: a runtime adapter table (argv, env, prompt input, parser, usage). `harness/src/scenario.cjs`: validate `runtime`. `harness/src/run.cjs`: per-host budget share and result keys. New scenarios: `markdown-exec-phase-chain-<host>` for each host, or one scenario with a `runtimes` list. |
+| Tests | `tests/harness.test.cjs`: per adapter, the argv builder and the parser against output captured from that CLI version, each shown to fail on the pre-change runner. A runtime-tagged scenario validates, and skips with its reason when the CLI is absent. A step whose usage cannot be priced is refused under `--max-usd`. |
+| Docs / ADR | `harness/README.md` (scenario table, how to run per host). An ADR-0047 amendment for D4 ("model steps take a runtime"), because it widens a recorded decision. No counts. |
+
+## Gate
+A tier-2 run of the first host's chain (Codex is suggested) must pass the same assertions as the Claude oracle. That means plans executed, the verification report written and the seed's tests passing. The run is recorded with the CLI version and its cost. Behind it, the adapter unit tests must fail on the pre-change runner.
+
+## Effort & risk
+- **Spend.**
+  - A Claude chain rep costs $3–6 (harness run of `2026-09-10`).
+  - The other hosts bill differently: Copilot in premium requests, Codex against a plan quota. So "dollars" may need an estimate from tokens, and the cap's meaning differs per host.
+  - Model tiers stay the owner's runs (ADR-0047 D2).
+- **Isolation versus auth.** M31 found the Codex gate writing into the real `CODEX_HOME`. A model step needs the user's credentials, and a scratch home does not have them. Copying a credential into a scratch directory is a security decision for the owner; the alternative is config-isolation flags on the real home.
+- **Unstable output shapes.** The CLIs' JSON is not a published contract. Pin the parser fixtures to the CLI version and re-capture them on upgrade, the way M25 and M31 pinned versions.
+- **Scope.**
+  - Only the markdown chain is portable.
+  - Antigravity cannot be run here (CLI absent).
+  - Gemini's headless model behaviour has not been exercised on this machine yet.
+- **Noise.** The Hugging Face cells are single attempts. PAN's chain already uses `--repeat 5`. Per-host results need the same repeats before any host-specific conclusion.
+
+## Open questions for the owner
+- Which host first? Codex is suggested: the most-exercised non-Claude gate (MI-071, M25, M31).
+- What spend cap per host per run, and how should a host billed in requests count against `--max-usd`?
+- Should credentials be copied into scratch homes, or the real home run with config overrides?
+- Should a host that fails the chain block a release (a gate), or only file a finding?

package/docs/specs/investigations/features/optimize-apply-regression-gate.md

@@ -0,0 +1,66 @@
+# Regression gate and rollback for `optimize apply`
+
+| | |
+|---|---|
+| Verdict | ENHANCE |
+| Priority | P3 (backlog) — its signal depends on [trace-failure-capture](trace-failure-capture.md) |
+| Size | M 4 points |
+| Area | learnings/optimize/experiment |
+| Runtimes | all five (the verbs are runtime-neutral; the signal they read is richest where the trace hook runs) |
+| First seen | `2026-09-28` — [digest](../digests/2026-09-28.md) |
+| Ledger | — |
+| Status | FILED `MI-061` |
+
+## Source trail
+- `2026-09-28` — Growing Harness (arXiv:2609.26760 v2): each proposed change to the agent program is kept only if success on a held-out gate set does not drop; otherwise the whole change sequence since the last accepted checkpoint is rolled back, including state. Without the gate, their gate success rose to 30% and then fell to 16%; with it, the best result was retained (one run per variant, 50 tasks). Repairing a window of several failures at once beat single-failure repairs by 8 points (same caveats) (source: arXiv:2609.26760, via the LLM brief of `2026-09-28`).
+
+## Why it matters for PAN
+`optimize apply` writes what a report recommends into the project, and the memory entries it writes are read by agents in every later run. A bad entry therefore degrades every future session, which is the regression the paper's gate exists to catch. Today PAN cannot tell whether an apply helped, and it cannot undo one mechanically.
+
+## Current state (evidence)
+- `pan-wizard-core/bin/lib/optimize.cjs:675` `applyReportRecommendations`: `memory` actions create a file (skipped if it exists), `memory_append` actions **append to an existing memory file**, and `note`/`planning_note` actions append to suggestion files. There is no check before or after. **SOURCE**
+- `optimize.cjs:752-761`: the log line in `optimization/applied.jsonl` records the timestamp, report name, counts and action **types** only: not the paths, not the appended text, not a pre-image. An append cannot be located afterwards to undo it. **SOURCE**
+- `optimize.cjs:800` `getOptimizeStats`: sums applied/skipped counts; nothing compares sessions before and after an apply. **SOURCE**
+- `pan-tools optimize` subcommands are `trace`, `learn`, `apply`, `list`, `stats` (dispatcher arm and `tests/fixtures/surface.json`). No `revert`. **SOURCE**
+- `optimize learn` analyses one trace session (`--session`), so each report reflects a single run. **SOURCE**
+
+## Proposal
+1. **Revertible applies.** `optimize apply` gives each run an `apply_id` and records, per action, the path, the kind (`created` / `appended`) and the exact appended text with a hash of the file after the write. `pan-tools optimize revert <apply_id>` deletes created files and removes appended spans, **refusing** any file whose current hash differs from the recorded one (a human edited it since), and reports what it refused.
+2. **A success-first verdict.** `pan-tools optimize verdict <apply_id>` compares trace sessions after the apply against an equal number before it: error events per spawn, verdict failures, retries, then tokens per spawn. It emits `kept`, `regressed` or `insufficient-data` (fewer than N sessions on either side; N in `.planning/config.json` under a new `optimize` section: `config.cjs` has none today). As in the paper, a cost drop never outweighs a rise in failures. The verdict recommends `revert`; it never reverts by itself.
+3. **Window, not single session (optional).** `optimize learn --sessions <n>` pools the last n sessions, so a recommendation must explain failures recurring across them rather than one run's accident.
+
+Honest limit: the paper gates on a fixed held-out task set. A PAN project has no such set; sessions before and after an apply do different work. The verdict is therefore advisory and says so in its output.
+
+## Implementation sketch
+| Layer | Change |
+|---|---|
+| Core module (`pan-wizard-core/bin/lib/`) | `optimize.cjs`: `apply_id`, per-action records, `revertApply()`, `applyVerdict()`, `--sessions` pooling in `generateLocalReport`; regenerate `tests/fixtures/module-surface.json` for the new exports |
+| Dispatcher (`pan-tools.cjs`) | `revert` and `verdict` arms inside `case 'optimize'`; extend the unknown-subcommand error text |
+| Installer / per-runtime (`bin/install-lib.cjs`) | none |
+| Commands / agents / workflows (markdown) | `commands/pan/optimize.md` and `workflows/optimize.md` document revert/verdict; `/pan:learn --apply` prints the `apply_id` |
+| Hooks / MCP | none (not an MCP verb: it writes the project) |
+| Tests | unit: append → revert restores bytes; edited-since file is refused; verdict thresholds; each shown to fail on the current module. `node scripts/test-surface.cjs` for the two new rows; the dispatcher-arms contract sees both arms |
+| Harness | not needed for the mechanics; a tier-1 scenario only once the trace signal exists |
+| Docs / ADR | `docs/CLI-REFERENCE.md` (no counts) |
+
+## Gate
+`tests/optimize.test.cjs` proves apply → revert round-trips byte-for-byte on a `memory_append`, refuses a hand-edited file, and fails on the current tree because `revert` does not exist.
+
+## Effort & risk
+- The revert half stands alone and is cheap. The verdict half is only as good as the trace signal, which is empty in the field today. Build it after, or together with, [trace-failure-capture](trace-failure-capture.md).
+- CRLF: appended spans must be matched EOL-tolerantly (the 3.31.0 lesson from `bytesMatchHashIgnoringEol`).
+
+## Open questions for the owner
+- Ship revert alone first (XS–S), and hold the verdict until the trace hook produces errors?
+- Is the optimize loop worth this investment at all, given the field shows it is barely used? The alternative is to fix the signal first and measure use before adding verbs.
+
+## Outcome (`2026-09-28`)
+
+Part 1 (revertible applies) is built on `feat/evidence-loop`:
+- every apply gets an `apply_id` with per-action records;
+- `optimize revert <apply_id|--last>` refuses a hand-edited file and reverts last-in, first-out per file, matching line endings tolerantly;
+- applying the same report twice now writes nothing the second time, where it used to append everything twice.
+
+Part 3 is built too: `optimize learn --sessions <n>` pools sessions.
+
+Part 2, the advisory before/after verdict, is deferred, which answers the owner's question: the loop is worth fixing (signal and safety) but not worth growing until it is used. The field showed 1 apply run in 14 projects. The apply records keep what the verdict needs, so it can be added without a migration. Trigger: apply runs in three or more projects.

package/docs/specs/investigations/features/reviewer-smell-baseline.md

@@ -0,0 +1,57 @@
+# Reviewer: a fixed code-smell baseline, each reported as a judgement call
+
+| | |
+|---|---|
+| Verdict | ENHANCE |
+| Priority | P3 (backlog) |
+| Size | S 2 points |
+| Area | agents |
+| Runtimes | all five (agent prompt only) |
+| First seen | `2026-10-10` — [digest](../digests/2026-10-10.md) |
+| Ledger | MI-106 |
+| Status | FILED MI-106 |
+
+## Source trail
+- `2026-10-10` — Matt Pocock's `code-review` skill always adds a fixed baseline of twelve smells from Fowler's *Refactoring* to whatever standards the repository documents. The smells are Mysterious Name, Duplicated Code, Feature Envy, Data Clumps, Primitive Obsession, Repeated Switches, Shotgun Surgery, Divergent Change, Speculative Generality, Message Chains, Middle Man and Refused Bequest. Two rules bind the baseline. A documented repository standard always overrides it. And each smell is reported as a labelled heuristic ("possible Feature Envy"), never a hard violation, and is skipped where tooling already enforces it. (source: `github.com/mattpocock/skills`, `skills/engineering/code-review/SKILL.md` at `49dd158`, `2026-10-09`, MIT)
+
+## Why it matters for PAN
+`pan-reviewer` reviews every phase's diff after execution. In a project that documents no standards, its quality checks reduce to a few size thresholds. The baseline gives it a stable, named vocabulary for design problems that thresholds miss.
+
+Several smells map onto failures PAN already guards against elsewhere:
+- **Speculative Generality** is the code-level form of the unrequested-work class (MI-031 lineage, `verify scope`).
+- **Shotgun Surgery** and **Divergent Change** are visible in a phase's diff and nowhere else.
+
+Reporting them as judgement calls keeps them below the verdict-changing tiers, so the meta-reviewer and the dispositions ledger handle them like any INFO finding.
+
+## Current state (evidence)
+- **The quality table is size-only.** It holds function length, nesting depth, dead imports, duplicate code (">10 identical lines across files", INFO) and new TODOs (`agents/pan-reviewer.md:63-70`). SOURCE.
+- **Conventions are discovered, never assumed.** The reviewer reads project instructions, `.agents/skills/` and existing code patterns, and is told not to assume any (`:39-44`). The baseline must keep that priority: the repository wins. SOURCE.
+- **The Spec axis is already separate.** The reviewer's Scope section covers unrequested work (`:72`), and `pan-verifier` checks the result against the phase goal. That is the skill's two-axis split, already in PAN's structure. SOURCE.
+
+## Proposal
+- **Add a "Design smells (baseline)" table** to the reviewer: the twelve smells, each with one line on what it is and one on the usual fix.
+- **Rules:**
+  - Always INFO and phrased "possible <smell>".
+  - Suppressed where a documented project standard endorses the pattern.
+  - Only for code the phase changed.
+  - Never a cause for a non-PASS verdict on its own.
+- **Speculative Generality defers to Scope.** It points at the Scope section, so the same code is not reported twice: unrequested behaviour stays `Unrequested`, and an unused abstraction is the smell.
+
+## Implementation sketch
+| Layer | Change |
+|---|---|
+| Core module / dispatcher / installer | none |
+| Commands / agents / workflows | `agents/pan-reviewer.md`: the table and its rules. `agents/pan-meta-reviewer.md`: one line saying baseline smells are judgement calls to downgrade or dismiss, never to escalate. |
+| Hooks / MCP | none |
+| Tests | a prompt-shape test that the reviewer carries the baseline as INFO with the "repository standard wins" rule, shown failing before the change |
+| Docs | `docs/AGENTS.md` pan-reviewer section |
+
+## Gate
+The prompt-shape test. A behavioural check is not worth paid runs for an INFO-only change.
+
+## Effort & risk
+- More INFO findings mean more noise in review reports. The meta-reviewer dedupes, and dispositions record dismissals.
+- The real risk is a smell creeping up to WARNING. The test pins INFO.
+
+## Open questions for the owner
+- All twelve, or only the ones visible in a diff?

package/docs/specs/investigations/features/trace-failure-capture.md

@@ -0,0 +1,69 @@
+# Failure capture in the trace hook, not in agent prompts
+
+| | |
+|---|---|
+| Verdict | ENHANCE |
+| Priority | P2 (this release cycle) |
+| Size | M 4 points |
+| Area | learnings/optimize/experiment · hooks |
+| Runtimes | Claude first (per-agent transcripts are proven there); Codex and Copilot also fire a subagent-stop event (`HOOK_EVENT_MAP` in `bin/install-lib.cjs`) but their transcript shape is unverified; Gemini and OpenCode register no subagent-stop hook |
+| First seen | `2026-09-28` — [digest](../digests/2026-09-28.md) |
+| Ledger | — |
+| Status | FILED `MI-060` |
+
+## Source trail
+- `2026-09-28` — Growing Harness (arXiv:2609.26760 v2): an agent program improves from task feedback only when each failure is recorded by the runtime as an execution trace (function calls, model and tool calls, errors) and tied to the code it passed through. Removing that function-level localisation halved final success in their ablation (36% → 18%, one run, 50 tasks). The paper's thesis is that recurring control belongs in code, not in model context (source: arXiv:2609.26760, via the LLM brief of `2026-09-28`).
+
+## Why it matters for PAN
+PAN's optimisation loop (`optimize trace` → `/pan:learn` → `optimize apply`) exists to learn from failures, but its failure signal is **delegated to the model**: the workflows instruct agents to run `pan-tools optimize trace log --type error …` at fixed points, and agents do not. This is the exact pattern the paper argues against: recurring control that the model is asked to redo every run. The field sweep of September 2026 measured the result: across the D-drive projects, trace events were almost all `decision/agent_completion`, and **zero** errors, gaps or corrections were ever recorded (project memory `field-telemetry-2026-09`, finding 4). The loop runs, but it has nothing to learn from. That also blocks the [optimize-apply regression gate](optimize-apply-regression-gate.md), which needs a failure signal to compare.
+
+Thesis fit: it moves an observation that can be made mechanically out of the prompt and into code, the "deterministic orchestration where control flow is knowable pre-run" pillar.
+
+## Current state (evidence)
+- `hooks/pan-trace-logger.js:830`: the SubagentStop hook writes one `type: 'decision', category: 'agent_completion'` event per spawn (tokens, duration, `exit_code`, model, command). **SOURCE**
+- `hooks/pan-trace-logger.js:880`: its only other event is a `redundancy` heuristic (same agent type twice with similar tokens). **SOURCE**
+- The hook already reads the per-agent transcript for token counts (v4 rows, per-agent `subagents/agent-<id>.jsonl`), and already walks `tool_result` records, but only to skip them when finding typed user turns (`hooks/pan-trace-logger.js:274`). **SOURCE**
+- Error, gap and correction events come only from `optimize trace log` calls written into workflow prose: `exec-phase.md`, `plan-phase.md`, `verify-phase.md`, `discuss-phase.md` and `optimize.md` all carry the instruction (`grep -c "optimize trace log" pan-wizard-core/workflows/*.md`). **SOURCE**
+- The field result (zero error events ever) is recorded in project memory, not re-measured today. **DOC**
+
+## Proposal
+Have the trace hook derive failure events from the subagent's own transcript, deterministically, when the spawn completes:
+
+- **`error / tool_error`**: a `tool_result` block with `is_error: true`, carrying the tool name and a normalised, truncated error line. One event per distinct (tool, first line) pair per spawn, so a retry loop does not flood the trace.
+- **`error / verdict_failed`**: the agent's final message carries a verdict PAN's own agents emit (the verifier's gaps verdict, the plan-checker's issues verdict). Match on the agent's own transcript, not on the parent's framed copy (Claude Code frames and indents subagent results since 2.1.277).
+- **`correction / retry`**: the same agent type re-spawned in the same phase after a `verdict_failed`. That is the escalation path MI-029 shipped (`resolve-model --attempt`), so the attempt number is the localisation.
+
+The localisation is `(command, phase, agent, tool)`, PAN's counterpart of the paper's "function the failure passed through". Keep the prompt-side `optimize trace log` instructions for what only the model can know (a knowledge gap, a surprise), and drop them where the hook now sees the event, so the prose shrinks.
+
+## Implementation sketch
+| Layer | Change |
+|---|---|
+| Core module (`pan-wizard-core/bin/lib/`) | `optimize.cjs`: accept the new categories in `generateLocalReport`'s error/gap grouping (`EVENT_TYPES` already has `error` and `correction`) |
+| Dispatcher (`pan-tools.cjs`) | none |
+| Installer / per-runtime (`bin/install-lib.cjs`) | none. The hook is already registered wherever `subagentStop` is non-null |
+| Commands / agents / workflows (markdown) | remove the `optimize trace log --type error` instructions the hook now covers; keep gap/surprise |
+| Hooks / MCP | `hooks/pan-trace-logger.js`: a pure `extractFailureEvents(transcriptRecords, agent)` beside the token reader; same dedupe signature scheme as the completion event |
+| Tests | fixture transcripts **emitted by a real run** (the fixture-from-emitted-output rule), one with a failing Bash tool call and one with a gaps verdict; assert events appear; show the test fails on the current hook; hook payload fixtures in `tests/` alongside the hooks-e2e suite |
+| Harness | a tier-0 scenario is not enough (no model step); a tier-1 scenario whose seed makes a test fail, asserting `optimize stats` shows a non-zero error count |
+| Docs / ADR | `docs/HOOKS.md` trace-logger section; no ADR (it restores the loop's stated design) |
+
+## Gate
+A tier-1 harness run on a seed with a deliberately failing test records at least one `error` event in `.planning/optimization/traces/*/trace.jsonl` without any agent calling `optimize trace log`, and the unit test built from that run's transcript fails against the pre-change hook.
+
+## Effort & risk
+- Transcript formats move. Claude's per-agent transcript is proven (the cost rebuild depends on it); Codex and Copilot payloads need a live probe before the extractor claims them. Ship Claude-only first and say so.
+- Noise: an `is_error` tool result is often a normal step (a grep with no match, a test run expected to fail in TDD). Categorise it rather than dropping it, and let `optimize learn` weigh repeats across spawns.
+- Privacy: error lines can carry paths or secrets. Truncate and redact with the same rules the cost ledger uses; everything stays in the project's `.planning/`.
+
+## Open questions for the owner
+- Keep or delete the workflow-prose `optimize trace log` calls the hook makes redundant?
+- Is the verdict match stable enough across PAN's agents, or should the verifier and plan-checker emit a machine line (for example a fenced JSON verdict) that the hook reads?
+
+## Outcome (`2026-09-28`)
+
+Built on `feat/evidence-loop`, designed in [evidence_loop_featureai.md](../../evidence_loop_featureai.md) (ADR-0049).
+- **Premise re-measured first.** No field project had ever recorded an error event, while subagent transcripts held 869 failed tool calls across 1,969 spawns. The write-up's claim that agents never call `optimize trace log` was only half right: they did, on success branches, and never on error branches.
+- **Verdicts.** Both open questions are answered in the spec. The judges emit a `pan-verdict` block, and a verb (`findings record`) reads it rather than the hook, because the verb works on every runtime and the workflow needs the answer. The verdict-path prose calls were removed.
+- **Tool failures.** The hook records them from per-agent transcripts on Claude Code, redacted, and `execution.error_pattern_learning: false` turns it off.
+- **Runtimes.** Codex and Copilot were checked: Copilot's documented payload has no per-subagent transcript, and its field names are now read.
+- **Gate met.** The tier-1 harness run `tool-error-capture` captured a real subagent's failing `npm test` through the installed hook ($0.50); the unit test built on the real record shape fails against the pre-change hook.