pi-session-mcp 0.8.1 → 0.8.2

package/docs/sprints/2026-10-13-s2-evidence.md

@@ -0,0 +1,431 @@
+# Sprint 2: evidence
+
+Base: main `b145f8c32cca76d23628ac8830069cee5ccbaece` (after Sprint 1, pull
+request #61). Date: 2026-10-13. Integration branch: `fix/s2-evidence`.
+Tracking: [#62](https://github.com/plagor44-ux/pi-session-mcp/issues/62) (R04),
+[#66](https://github.com/plagor44-ux/pi-session-mcp/issues/66) (R13),
+[#67](https://github.com/plagor44-ux/pi-session-mcp/issues/67) (R14),
+[#68](https://github.com/plagor44-ux/pi-session-mcp/issues/68) (handshake
+offering), [#63](https://github.com/plagor44-ux/pi-session-mcp/issues/63)
+(R16), [#64](https://github.com/plagor44-ux/pi-session-mcp/issues/64) (R20),
+[#65](https://github.com/plagor44-ux/pi-session-mcp/issues/65) (R23) and the
+Sprint 1 follow-up [#60](https://github.com/plagor44-ux/pi-session-mcp/issues/60).
+Planning sources: the [roadmap](../roadmap.md) and the
+[post-0.8.1 audit](../reviews/2026-10-13-post-0.8.1-audit.md), whose Sprint 2
+section defines the exit gate. The [Sprint 1 file](2026-10-13-s1-correctness.md)
+holds the closure of the previous sprint.
+
+This file tracks the state of Sprint 2. It records what is planned, what code
+exists and what is proven. A status of `implemented` or `verified` is set only
+together with evidence in the sections below. The statuses and evidence below
+are the final state of the sprint, recorded before its pull request
+[#69](https://github.com/plagor44-ux/pi-session-mcp/pull/69) was merged as
+`0aae639`; the merge closed #60, #62, #63, #64, #65, #66, #67 and #68. Its
+final-head CI passed; the later main CI failure on `macos-15` is recorded in
+the [Sprint 3 file](2026-10-13-s3-live-usage.md#main-ci-on-the-sprint-2-merge).
+
+## Scope
+
+In scope:
+
+- R04: the orchestration skill classifies every documented `pi_session_start`
+  error code ([#62](https://github.com/plagor44-ux/pi-session-mcp/issues/62)).
+- R13: the multi-session acceptance harness binds the contents of the loaded
+  Pi extensions ([#66](https://github.com/plagor44-ux/pi-session-mcp/issues/66)).
+- R14: the harness binds the Codex skill sources, a prerequisite for the live
+  discovery check R07
+  ([#67](https://github.com/plagor44-ux/pi-session-mcp/issues/67)).
+- The handshake offering: the harness offers its test skill only to the task
+  that needs it, a prerequisite for a new Codex abort run in R06
+  ([#68](https://github.com/plagor44-ux/pi-session-mcp/issues/68)).
+- #60: the dry-run of a pending skill transaction predicts a retained
+  intermediate ([#60](https://github.com/plagor44-ux/pi-session-mcp/issues/60)).
+- R16: CI runs the tests with strict unhandled rejections
+  ([#63](https://github.com/plagor44-ux/pi-session-mcp/issues/63)).
+- R20: delivery-status notes on the two historical plans
+  ([#64](https://github.com/plagor44-ux/pi-session-mcp/issues/64)).
+- R22: proposed replacement text for the ChatGPT project instructions, which
+  live outside this repository (see the appendix).
+- R23: acceptance and versioning wording
+  ([#65](https://github.com/plagor44-ux/pi-session-mcp/issues/65)).
+- A current-source Pi SDK canary on the frozen candidate with
+  `scripts/pi-sdk-canary.sh`, without moving the pin.
+- The documentation that these changes directly affect.
+
+R19 and R21 were delivered in Sprint 1 (R19 in the Sprint 1 pull request, R21
+by [#58](https://github.com/plagor44-ux/pi-session-mcp/pull/58)) and are not
+repeated here.
+
+Out of scope: the Sprint 3 and 4 live runs (R05 to R11, R12, R17, R18), a
+Level 3 cycle, macOS live runs, new features, dependency or Pi SDK updates, a
+version bump and any change to the MCP contract. The harness changes concern
+the maintainer script, which is not part of the npm package, not the package
+runtime.
+
+R-identifiers are planning identifiers, not GitHub issue numbers.
+
+## Delivery
+
+One integration branch and one pull request against main, with separate
+commits per issue. The work runs in separate worktrees with their own file
+ownership, and an integrator combines them. No merge, tag or publication is
+part of this sprint.
+
+| Strand | Branch | Owns |
+| --- | --- | --- |
+| G | `s2/g-dryrun` | #60 fix in `src/setup-skills.ts` |
+| I | `s2/i-dryrun-tests` | #60 counterexamples in `test/setup-skills-dry-run-recovery.test.ts` |
+| J | `s2/j-skill` | #62 in `skills/pi-session-orchestration/SKILL.md` and `test/orchestration-skill-start-codes.test.ts` |
+| H | `s2/h-harness` | #66, #67, #68 in `scripts/multi-session-acceptance.mjs` and its test |
+| F | `s2/f-docs` | Sprint tracking, plan delivery notes, wording and changed-behavior documentation |
+| Integrator | `fix/s2-evidence` | `.github/workflows/ci.yml` (#63), the harness sections of acceptance, integration, final documentation and the pull request |
+
+One owner edits the harness. Writing strands never share a file.
+
+## Status legend
+
+| Status | Meaning |
+| --- | --- |
+| planned | Not started |
+| in_progress | Work has started; no result may be claimed yet |
+| implemented | The code or text exists; the acceptance criteria are not yet met with evidence |
+| verified | The named acceptance criteria are met and the evidence is recorded below |
+| blocked | A named blocker prevents progress |
+| not included | Deliberately outside this sprint, with an entry condition |
+
+## Tasks
+
+| Task | Issue | Owner strand | Branch | Depends on | Status |
+| --- | --- | --- | --- | --- | --- |
+| R04 orchestration skill classifies every start code | [#62](https://github.com/plagor44-ux/pi-session-mcp/issues/62) | J | `s2/j-skill` | None | verified |
+| R13 bind loaded extension package contents | [#66](https://github.com/plagor44-ux/pi-session-mcp/issues/66) | H | `s2/h-harness` | None | verified |
+| R14 bind all Codex skill sources | [#67](https://github.com/plagor44-ux/pi-session-mcp/issues/67) | H | `s2/h-harness` | None; prerequisite for R07 | verified |
+| Handshake offered only where a task needs it | [#68](https://github.com/plagor44-ux/pi-session-mcp/issues/68) | H | `s2/h-harness` | None; prerequisite for a new Codex abort run (R06) | verified |
+| Dry-run predicts a retained intermediate | [#60](https://github.com/plagor44-ux/pi-session-mcp/issues/60) | G (fix), I (counterexamples) | `s2/g-dryrun`, `s2/i-dryrun-tests` | Sprint 1 merge (done) | verified |
+| R16 strict unhandled rejections in CI | [#63](https://github.com/plagor44-ux/pi-session-mcp/issues/63) | Integrator (`ci.yml`), F (command in `CONTRIBUTING.md` and `docs/releasing.md`) | `fix/s2-evidence`, `s2/f-docs` | None | implemented (takes effect in the final-head CI of the pull request) |
+| R20 delivery status of the historical plans | [#64](https://github.com/plagor44-ux/pi-session-mcp/issues/64) | F | `s2/f-docs` | None | verified |
+| R23 acceptance and versioning wording | [#65](https://github.com/plagor44-ux/pi-session-mcp/issues/65) | F | `s2/f-docs` | None | verified |
+| R22 ChatGPT project instructions | None (outside this repository) | Operator | None | An operator action; this repository only proposes a text | blocked (external) |
+| R15 explicit unverified evidence for an unreadable client start time | None | None | None | See the entry condition below | not included |
+| Current-source Pi SDK canary on the frozen candidate | None | Integrator | `fix/s2-evidence` | Frozen candidate; the pin does not move | verified (SDK 1.2.0 compatible; pin unchanged) |
+| Sprint tracking, roadmap, changelog, S1 closure | None | F, then the integrator | `s2/f-docs`, `fix/s2-evidence` | None | verified |
+
+Status as of 2026-10-13, after integration, four native gate runs, the Pi
+SDK canary, Level 2 and two independent reviews (see [Evidence](#evidence)).
+"verified" refers to the acceptance criteria of each task below. It does not
+include the final-head CI matrix of the pull request or the maintainer's
+merge decision; the Sprint 2 exit gate stays open until both exist.
+
+R22 is blocked because the ChatGPT project instructions are not stored in this
+repository and the integrator cannot change them. The
+[appendix](#appendix-proposed-chatgpt-project-instructions-r22) holds a
+proposal; this repository does not apply it.
+
+R15 is not included. An unreadable client start time directly after spawn
+cannot produce a false pass: the run fails with `noServerObserved`, which is a
+misleading code but still a failing one. Entry condition: a run that passes
+while the start time was unreadable, or a demonstrated need for an explicit
+unverified code.
+
+## Behavior contracts
+
+These describe the intended change. They are contracts, not results; none is
+verified until the evidence section says so.
+
+- #60: for a pending version 2 skill transaction, `--dry-run` reports
+  `skill_recovery_retained` (status `divergent`, exit 1) whenever `--apply`
+  would retain an intermediate, using the same read-only classification as the
+  recovery. Otherwise it still reports `skill_recovery_planned`. A dry-run
+  never writes.
+- #62: the orchestration skill classifies every documented `pi_session_start`
+  code as operator-needed, transient or unknown, by the code alone. A code that
+  is not listed is unknown. The `startup_failed` diagnostics mean that no MCP
+  server is available.
+- #63: CI runs the tests with `NODE_OPTIONS=--unhandled-rejections=strict` on
+  the existing matrix and nothing else changes in the workflow.
+
+## Acceptance criteria
+
+### R04 / #62: start-code classification
+
+- The skill lists every start code documented in the
+  [tool contracts](../tool-contracts.md) in exactly one class: operator
+  needed, transient or unknown.
+- A test fails if a documented start code is missing from the skill or appears
+  in two classes.
+- The skill states that classification uses the code only and that unlisted
+  codes are unknown. Provider free text is never classified.
+- Documentation that mirrors the skill's rules is aligned.
+- No runtime, MCP contract or error-code change.
+
+### R13 / #66: bind loaded extension contents
+
+- `prepare` resolves the agent directory once, records it privately, and the
+  run's MCP server receives it explicitly as `PI_CODING_AGENT_DIR` for both
+  clients.
+- When the run uses extensions, the bound resources include each extension's
+  package root tree and the agent directory's npm installation, dependencies
+  included, under path-free keys. Runs without extensions get a stable `none`
+  value.
+- A change to any of them between `prepare`, launch and the end of the run
+  reports `resourcesChanged` and fails or refuses the run.
+- Tests change an extension file, a dependency file and the npm lockfile after
+  `prepare` and observe `resourcesChanged` before launch and after a run.
+- Tests never read or write the operator's real home or Pi directory. Results
+  and evidence stay path-free.
+- [Acceptance](../acceptance.md) describes the binding.
+
+### R14 / #67: bind all Codex skill sources
+
+- `prepare` resolves `CODEX_HOME` once and the Codex controller runs with
+  exactly that value.
+- `$CODEX_HOME/skills` (including the bundled `.system` skills),
+  `/etc/codex/skills`, other skills in `~/.agents/skills` and `.agents/skills`
+  directories in ancestors of the controller directory are bound under
+  path-free keys, as a tree hash or a stable `missing` value.
+- A differing `pi-session-orchestration` copy in any source refuses a Codex run
+  before launch; an equal copy or none is allowed.
+- Evidence states per client whether the controller skill was an installed copy
+  (Claude Code) or a run-scoped link (Codex), with its tree hash, without
+  paths. Installed and linked skills are distinguishable.
+- Tests cover a change in each source, a differing skill in each source, an
+  equal copy and a different `CODEX_HOME` at launch, and never touch the
+  operator's real home, `CODEX_HOME` or `/etc`.
+- [Acceptance](../acceptance.md) lists the sources and the refusal rule.
+
+### Handshake offering / #68
+
+- Two coding profiles: `accept-code` with exactly `implementation` for every
+  task, and a separate profile with exactly `implementation` and `handshake`,
+  used only for the `slug` session.
+- The controller prompt names the profile of each task.
+- The skill criterion checks both skill lists and that only the `slug` session
+  used the handshake profile; a run in which another task used it fails.
+- The scope oracle and `ALLOWED_CHANGES` stay unchanged and strict. This is not
+  a wider allowed-file set. Earlier failed attempts stay recorded unchanged.
+- [Acceptance](../acceptance.md) describes the two profiles.
+
+### #60: dry-run prediction
+
+- For each retained case of the recovery tests (partial, modified or added-to
+  trash; modified backup; foreign staging), `--dry-run` reports
+  `skill_recovery_retained` and changes nothing.
+- For each settleable case, `--dry-run` still reports `skill_recovery_planned`.
+- Classification is shared with the recovery. No second rule set, no new
+  persisted state, no name-pattern search and no new report code.
+- [Skills](../skills.md#interrupted-updates) and the
+  [setup design](../client-setup-design.md) no longer describe the dry-run as
+  optimistic.
+
+### R16 / #63: strict CI
+
+- `ci.yml` sets the strict option for the test step only, on every platform of
+  the existing matrix, without new matrix entries.
+- The CI run of the change passes on Linux, macOS 15 and macOS 26.
+- `CONTRIBUTING.md` and [Releasing](../releasing.md) name the same strict
+  command.
+
+### R20 / #64: plan delivery status
+
+- Both plans carry a dated delivery-status note with pull request references
+  and the actual remainder, and state that their checkboxes are historical.
+- No claim of acceptance beyond the recorded evidence; nothing is
+  reimplemented; relative links resolve.
+
+### R23 / #65: wording
+
+- No passage presents targeted two-session evidence as Level 3.
+- One place defines the package version, the MCP tool contract and the setup
+  report `schemaVersion` ([tool contracts](../tool-contracts.md#versions)), and
+  other places link to it.
+- Recorded results are unchanged; historical records get at most a clarifying
+  note.
+
+### Sprint 2 exit gate
+
+From the audit, with the issue-level criteria above:
+
+- Changing a bound skill or extension after preparation invalidates a run.
+- Installed and linked skills are distinguishable in the evidence.
+- Original failed attempts remain recorded.
+- Every remainder has an issue or an explicit entry condition (R15 above, R22
+  as an operator action, the S3 and S4 items in the roadmap).
+- No new runtime feature phase.
+
+## Evidence
+
+Recorded by the integrator on 2026-10-13 on native Linux (x64, kernel 7.2,
+Btrfs checkout, tmpfs temporary root), Node.js `26.11.1`, npm `12.2.0`, Codex
+CLI `0.162.1`, Claude Code `2.1.296`. No provider or model was used by any
+check. A result for one commit is not evidence for another; each row names
+the commit it ran on.
+
+### Before the fix (base `b145f8c`)
+
+| Issue | Check | Result |
+| --- | --- | --- |
+| #60 | `test/setup-skills-dry-run-recovery.test.ts`, written against the contract | Exit 1: 14 of 23 failed, each at the dry-run assertion (received `skill_recovery_planned`/`planned` where `skill_recovery_retained` was required); the 9 settleable cases passed. |
+| #62 | `test/orchestration-skill-start-codes.test.ts` against the base skill (scratch copy) | Failed at suite level because the base skill has no classes at all (`expected one class bullet for Operator needed, found 0`); per-code assertions did not run. |
+| #66, #67, #68 | New harness tests | Against the base harness most new tests fail because the functions they test do not exist yet, which proves little. The harness author showed instead, with single-fix mutants of the new harness (client stand-ins first on `PATH`), that each new test fails when its own fix is removed. These mutant runs are author-reported and were not repeated by the integrator; both independent reviewers judged the new assertions to target the defects. |
+
+### Native gate
+
+| Head | Runtime candidate | Result |
+| --- | --- | --- |
+| `29d54b0` | `e5af851` | All steps exit 0; 54 files; 1208 passed, 6 skipped |
+| `a3b47af` | `1f32de7` (after the first review round) | All steps exit 0; 54 files; 1232 passed, 6 skipped |
+| `9e67654` | `94f8099` (after the second round) | All steps exit 0; 54 files; 1278 passed, 6 skipped |
+| `07f56b6` | `923a393` (final) | All steps exit 0; 54 files; 1300 passed, 6 skipped |
+
+Every run used `npm ci`, `npm run typecheck`, `npm run build:cli`,
+`NODE_OPTIONS=--unhandled-rejections=strict npm test`, `npm run build`,
+`npm audit`, `npm audit --omit=dev` (0 vulnerabilities each time),
+`git diff --check b145f8c HEAD` and a clean `git status --short`. The counts
+reconcile with Sprint 1's 1127 cases: the new #60, #62 and harness tests account
+for every addition, and the six skips are the same as in Sprint 1.
+
+### Pi SDK canary
+
+`scripts/pi-sdk-canary.sh` on `29d54b0` (runtime `e5af851`): pinned `1.1.0`,
+latest published `1.2.0`. Typecheck passed; the 12 SDK-facing test files
+passed (311 passed, 1 skipped). The script has a built-in single retry for
+known load-sensitive tests; no test was retried. `package.json` and
+`package-lock.json` stayed byte-identical and `npm ci` restored the pinned
+tree. Later commits change no SDK-facing test and change `src/` only by one
+comment line. This is a compatibility observation, not an SDK update.
+
+### Level 2
+
+`scripts/setup-acceptance.mjs`, real Codex and Claude Code in disposable
+homes, no model:
+
+| Build | Result |
+| --- | --- |
+| Runtime `e5af851` (`dist` JS tree `f84fc446…`) | 160/160, exit 0 |
+| Runtime `1f32de7` (`dist` JS tree `f85bf73e…`; differs from the previous build only by one comment in `setup-skills.js`) | 160/160, exit 0; `codex:user`, `claude-code:user`, `project` and `local` 9/9 each; real registrations unchanged, no remaining process, no path leak, build unchanged, disposable directory removed |
+
+Every later commit changes only the maintainer harness, tests and
+documentation; the final head builds the same `dist` tree `f85bf73e…`, which
+binds this Level 2 result to it.
+
+### Package and links
+
+On `07f56b6`, `npm pack --dry-run --json` exited 0: 133 files, 248,627 bytes, both sprint files included, and no `src/`, `test/`, lockfile, implementation plan or maintainer harness script. A relative link and heading-anchor check of every changed Markdown file outside the historical plans, plus README and the audit, found 194 links and no problem on the final documentation; the historical macOS plan keeps one pre-existing link inside its drafted README text.
+
+### Real-client incident during development
+
+While developing #66, an unguarded scratch mutant of the harness test started
+the real `claude -p` three times before the test file installed its client
+stand-ins. The test timeout ended each process after about five seconds. No
+session transcript was written and no process remained; a model request in
+flight at that moment cannot be excluded. The three empty project directories
+it left in the operator's Claude Code data were removed. Afterwards the harness
+tests put stand-in `claude` and `codex` executables first on `PATH`, and the
+integrator checked every harness and full-suite run for client sessions whose
+recorded working directory lies in a test root, and for client processes:
+none were found. One unrelated Codex session of another project, started
+concurrently, was identified by its recorded working directory and is not
+part of this work.
+
+## Independent review
+
+Two read-only reviewers who did not write the change reviewed the frozen
+runtime diff `b145f8c..e5af851` and the fix deltas:
+
+| Reviewer | Round | Verdict | Outcome |
+| --- | --- | --- | --- |
+| Claude family | 1 | approve with nits | Hermetic system-skill path in a test driver, controller-skill identity by declared name, the `CODEX_HOME` upgrade note: fixed. A claim that recovery now retains a proven trash was checked against the code and rejected; a pinning test (`4f67188`) records the actual behavior. |
+| OpenAI family (Pi `review` profile) | 1 | changes requested | Content behind nested links, `.system` controller skills, hermeticity of `prepare`, equal symlinked copies: fixed in `1db846f`, `850f779`, `1f32de7`. |
+| OpenAI family | 2 | changes requested | Incomplete digests treated as stable, YAML comments in the skill name, a blocking header read: fixed in `0386d5d`, `fb79f23`, `94f8099`. |
+| OpenAI family | 3 | changes requested | Indented or flow YAML headers and symlinked candidate skill roots: fixed in `b1a6b0f` and `61d1721`; the FIFO tests now run under a watchdog (`923a393`). |
+| OpenAI family | 4 | approve with nits | No blocker or major finding; the remaining minor limits are listed below. |
+
+The package runtime (#60, #62, #63) passed both reviewers without a required
+change in every round; all requested changes concerned the maintainer
+acceptance harness.
+
+### Remaining limits recorded by the reviews
+
+- Only skills found by directory name or by a `SKILL.md` frontmatter `name`
+  are checked; skills that Codex might load by other means (other file names,
+  plugin caches) are not covered. YAML merge keys (`<<`) are not read as a
+  name source; it is not established that Codex honors them.
+- Exotic but valid frontmatter in an operator's own skills is conservatively
+  indeterminate and refuses a Codex run.
+- Bound trees are digested by reading whole files; the 200,000-entry cap is
+  neither a byte nor a time budget. An unreadable or too large tree fails
+  closed with `resourcesUnbound`.
+- The Codex controller directory's own `.agents/skills` is bound only through
+  the run-scoped link; Codex writing its `.system` skills during a run shows as
+  `resourcesChanged` and fails safe.
+
+## Open gates
+
+- Final-head CI on Linux, macOS 15 and macOS 26 for the pull request, which
+  also exercises the strict test step of #63 for the first time; its result
+  belongs to the pull request.
+- The maintainer's merge decision.
+- R22: an operator updates the ChatGPT project instructions with the
+  [proposed text](#appendix-proposed-chatgpt-project-instructions-r22).
+- R15: not included until its entry condition holds.
+- Sprint 3 and 4 as in the [roadmap](../roadmap.md): the ten-task Level 3
+  cycle (R05), a new Codex abort run (R06), live discovery of the
+  setup-installed Codex home skill (R07), a third and a read-only worker (R08,
+  which needs the stricter `workerSkills` rule extended for a read-only
+  session), the profile actually intended for use (R09) and current macOS
+  Level 2 on real hardware (R10, R11).
+
+No release is authorized by this sprint. A 0.8.2 patch would carry #59, #56,
+#60 and the skill change of #62 under an unchanged MCP contract; it needs its
+own release scope and pre-release acceptance.
+
+## Next step
+
+When final-head CI passes and the maintainer has merged the one pull request,
+merging closes #60, #62, #63, #64, #65, #66, #67 and #68, and Sprint 3 (live
+acceptance) can begin under the entry conditions in the
+[roadmap](../roadmap.md). Its dependent runs use the bindings and the
+handshake profile split delivered here. Before the first Codex run, start
+Codex once with the intended `CODEX_HOME` after any upgrade, so its bundled
+skills are installed before `prepare`.
+
+## Appendix: Proposed ChatGPT project instructions (R22)
+
+This is a proposal for the operator to paste into the ChatGPT project
+instructions. This repository does not apply it and cannot check that the
+instructions are updated. It describes the state on 2026-10-13.
+
+```text
+Project: Pi Session MCP
+
+Current repository: plagor44-ux/pi-session-mcp. It is the successor of
+pi-control. pi-control is archived; its final pull request was #79. Do not
+work in pi-control and do not treat its history as the current plan.
+
+Current release: package and release 0.8.1 on npm (pi-session-mcp). It uses
+Pi SDK 1.1.0 and MCP SDK 2.3.1. Local stdio, in-memory state, one trusted
+local peer.
+
+Decisions that bind current work (docs/adr, in this order):
+0001 stdio first; 0002 embedded Pi SDK; 0003 post-MVP maintenance; 0004 client
+contract, not client version; 0005 macOS setup platform; 0006 Pi extension
+allowlist (exact npm:<name>@<version> entries per execution profile); 0007
+profile-selected skills and the setup-installed controller skill.
+
+The project is in maintenance mode (ADR 0003): focused fixes and upstream
+drift, no new feature phase. A change to the public MCP contract needs a new
+minor release and its own decision.
+
+Current priorities come only from docs/roadmap.md and the sprint files in
+docs/sprints/ (together with docs/reviews/2026-10-13-post-0.8.1-audit.md for
+the R01-R23 backlog). Read them before proposing work. Roadmap windows are
+proposals and shift with their gates.
+
+Historical gates are not active priorities: the v0.3.0 release sequence and
+the former pi-control issues #11-#14 are historical. Bind any
+acceptance to the current build, not to those gates.
+
+Rules: state only what was run; provider-free tests do not establish live
+client or model behavior; targeted two-session checks are not a Level 3
+cycle; no secrets, absolute paths or prompts in records.
+```