pi-crew 0.11.6 → 0.11.8

package/.githooks/pre-commit

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+# pi-crew pre-commit hook — block lockfile-drift commits locally.
+#
+# Catches the "bump a devDependency in package.json but forget `npm install`"
+# class of CI failure BEFORE it is pushed. Runs the fast (<50ms, no install)
+# check-lockfile-sync.mjs gate only when package.json or package-lock.json is
+# staged, so it adds zero overhead to unrelated commits.
+#
+# Activate once per clone:  git config core.hooksPath .githooks
+# Bypass for a WIP commit:  git commit --no-verify   (CI will still catch it)
+
+set -euo pipefail
+
+# Only run when the manifests are staged (cheap path for unrelated commits).
+if git diff --cached --name-only -- package.json package-lock.json | grep -q .; then
+  echo "[pre-commit] checking package.json ↔ package-lock.json sync…"
+  if ! node scripts/check-lockfile-sync.mjs; then
+    echo "[pre-commit] ✋ commit blocked: package-lock.json is out of sync."
+    echo "[pre-commit]    Fix: run 'npm install' and 'git add package-lock.json', or use --no-verify to bypass (CI still gates)."
+    exit 1
+  fi
+fi

package/.githooks/pre-push

@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# pi-crew pre-push hook — broaden the local release gate (D3).
+#
+# The 2026-10 CI-red recurrence: pushes to origin/main ran only
+# test:critical (~120 tests) locally, while CI runs the FULL unit suite —
+# three waves in a row shipped red. This hook closes that gap BEFORE the
+# push leaves the machine.
+#
+# Scope:
+#   push to refs/heads/main → check:conflict-markers + typecheck, then the
+#                            FULL unit suite (npm run test:unit — the same
+#                            gate CI runs); any failure blocks the push
+#   any other ref          → fast gates only (check:conflict-markers +
+#                            typecheck)
+#
+# Activated by `npm install` (prepare → scripts/install-hooks.mjs sets
+# git config core.hooksPath .githooks) or manually:
+#   git config core.hooksPath .githooks
+# Emergency bypass:  git push --no-verify   (CI still gates main).
+
+set -euo pipefail
+
+cd "$(git rev-parse --show-toplevel)"
+
+pushes_main=0
+while read -r local_ref local_sha remote_ref remote_sha; do
+	if [ "$remote_ref" = "refs/heads/main" ]; then
+		pushes_main=1
+	fi
+done
+
+if ! npm run check:conflict-markers; then
+	echo "[pre-push] ✋ push blocked: conflict markers present."
+	echo "[pre-push]    Fix: resolve the markers, or bypass with 'git push --no-verify' (CI still gates)."
+	exit 1
+fi
+if ! npm run typecheck; then
+	echo "[pre-push] ✋ push blocked: typecheck failed."
+	echo "[pre-push]    Fix the errors above, or bypass with 'git push --no-verify' (CI still gates)."
+	exit 1
+fi
+
+if [ "$pushes_main" = "1" ]; then
+	echo "[pre-push] push to main: running the FULL unit suite (test:unit)…"
+	# Capture the full log so a flake's failing test is identifiable even when
+	# the push output is truncated by tooling (lesson 2026-10-07: three hook
+	# blocks lost their failure detail to a trailing `| tail`).
+	UNIT_LOG="${TMPDIR:-/tmp}/pi-crew-prepush-unit.log"
+	if ! npm run test:unit 2>&1 | tee "$UNIT_LOG"; then
+		echo "[pre-push] ✋ push blocked: test:unit failed (full log: $UNIT_LOG)."
+		echo "[pre-push]    Fix the failures above, or bypass with 'git push --no-verify' (CI still gates)."
+		exit 1
+	fi
+fi
+
+echo "[pre-push] gates green."

package/CHANGELOG.md

@@ -1,5 +1,221 @@
 # Changelog
 
+## [0.11.8] — Pi 1.0.4 adoption + RPC/surface hardening + battery-proven stability (2026-10-07)
+
+### Added — deep-learn polish batch (docs/reviews/pi-1.0.0-deep-learn-r3-2026-10-03.md R3 matrix)
+
+- **RPC worker transport wired (P2-3 integration)** — after a live-fire probe against a real
+  `pi --mode rpc` (24 ui-requests drained by the drain policy; dialog auto-answered `cancel`
+  with no deadlock; steer honored at 53 ms RTT), the `runtime.workerTransport: "rpc"` path now
+  dispatches real RPC workers with worker-cap parity, a pre-abort short-circuit, and a guarded
+  stdio fallback. Still experimental: minimal input mapping, raw model passthrough, ephemeral
+  sessions (see src/runtime/rpc/README.md).
+- **Skill advertisement dedup (R3-23)** — the worker spawn path no longer duplicates the host's
+  `<available_skills>` block with pi-crew's own per-skill index; measured ~232–284 tokens/worker
+  (~65% of the block) saved, ~1.9k tokens per 8-role wave. Live-session/scaffold keep the full
+  block.
+- **Foreign-pi orphan scan (R3-16)** — `doctor --zombies` now WARN-lists orphaned pi processes
+  via `AI_AGENT`/`PI_CODING_AGENT` markers (exact-value match, argv pi-binary gate, self/ancestor
+  exclusion). Report-only by design — never a kill candidate.
+- **Worker session labels (R3-15)** — headless workers spawn with `--name crew-<taskId>` for
+  human-friendly session pickers.
+- **Live-session scoped models (R3-12)** — live sessions bind `scopedModels` to the configured
+  fallback chain (explicit-chain-only; user overrides preserved — scope is visibility, not gate).
+- **OSC-8 dock link (R3-9)** — the crew dock subject is a `file://` hyperlink to the latest run's
+  artifacts dir; degrades to plain text on old hosts, escape-safe under truncation.
+
+### Changed
+
+- **Local pre-push release gate broadened via committed `.githooks/` (D3)** — a new
+  `pre-push` hook (activated by the `prepare` script on `npm install`, or manually via
+  `git config core.hooksPath .githooks`) runs `check:conflict-markers` + `typecheck` on every
+  push, and additionally the FULL `test:unit` suite when pushing to `refs/heads/main` — the
+  same gate CI runs — so a leader's local fast-gate-only push can no longer ship red.
+  Emergency bypass: `git push --no-verify` (CI still gates). Documented in docs/publishing.md.
+- **`runtime.workerTransport` config plumbed to the run-worker seam (D1/DR2)** — the config key
+  is no longer parse+emit-only dead: callers carrying runtimeConfig (child-executor,
+  run-coalesced-task-group) thread it to `runWorker`, with the established precedence
+  env > config > default stdio (`PI_CREW_WORKER_TRANSPORT` still wins when set; invalid values
+  fail safe to stdio). The three stale "not-implemented" doc blocks (config-schema.ts,
+  config/types.ts, env-vars.ts) now describe the wired state; the F19-1 comment records that
+  emission without a consumer is still dead. dynamic-workflow-context and goal-evaluator have
+  no runtimeConfig and deliberately keep default stdio.
+- `setupRenderLoop` extracted byte-identically into `src/extension/registration/render-loop.ts`
+  (lifecycle-handlers.ts 1347→897 lines) (QW#2).
+- `formatDuration` name collision resolved with domain names (`formatTaskElapsed`,
+  `formatRelativeCompact`) — zero output-format changes (QW#5).
+- wc-gate now scans **all of `src/`** (527 files; 0 violations, max 1908) (QW#3).
+- f19-config-parity test now auto-derives runtime keys 3-way (types ↔ schema ↔ parser) — no hand
+  lists; mutation-verified (parse-dead and phantom keys both fail loudly).
+- IME-safe cursors: hand-written text inputs emit pi-tui `CURSOR_MARKER` (R3-6).
+- Bundle benchmark comment refreshed to the re-measured ~36% cold-start gap (QW#6).
+- R3-10 (withFileMutationQueue) skipped with evidence: pi-crew's flock-based mutation locks
+  already cover the named paths; the SDK helper is in-process-only — double-lock for no gain.
+
+### Added — Pi 1.0.4 adoption wave (see docs/reviews/pi-1.0.4-upgrade-notes-2026-10-06.md)
+
+- **devDeps on the pi SDK 1.0.4 (U1, P0)** — all four `@earendil-works` devDeps bumped
+  `^1.0.0`→`^1.0.4` with a lockfile refresh. pi-coding-agent@1.0.4 direct-pins
+  `brace-expansion 5.0.12`, closing the accepted dev-scope 5.0.9 finding: the top level now
+  hoists 5.0.12 and the pi subtree no longer nests a copy.
+- **Thinking level `max` (U4)** — `THINKING_LEVELS` carries all 7 pi levels (was 6);
+  frontmatter `thinking: max` was previously rejected by `isValidThinkingLevel` even though
+  pi's CLI documents it.
+- **MCP cut for parity spawns (U2-lite)** — pi 1.0.4 no longer cuts MCP under `--tools`, so
+  parity (NON-hermetic) spawns fold `mcp__*` into their single `--exclude-tools` value
+  (merged comma-sep with frontmatter `disallowedTools`; pi's strict parser rejects duplicate
+  flags). Hermetic defaults gain no new flag — `--no-extensions` already kills built-in MCP
+  on 1.0.4. Cross-version-safe: on pre-1.0.4 hosts the pattern matches no literal tool name.
+- **Classifier seam, dormant (P2-1)** — `runtime.classifierEnabled` (default **false**) +
+  `runtime.classifierModel` (default `opencode/jev-1.13-free`; env `PI_CREW_CLASSIFIER_ENABLED`
+  / `PI_CREW_CLASSIFIER_MODEL` beats config) land with full config ceremony (schema → types →
+  validation → env registry → snapshot test). Host-process `classifyBool()` is never-rejects:
+  missing registry / no classifier available / classify throw / `stopReason:"error"` all return
+  the caller-supplied fallback decision with log-once + `crew.classifier.calls_total` counters —
+  never an exception up to the orchestrator. Spike consumer: child-executor retry-triage
+  classifies transient-vs-permanent before burning a re-attempt (event `task.retry_triage`,
+  metric `crew.classifier.decisions_total`); default-off never touches the registry (zero
+  behavior change, test-pinned). Live-fire probe for post-credential runs:
+  `scripts/probe-classifier.mjs`.
+- **Pi version floor + `--no-mcp` on parity spawns (U9)** — new `probePiVersion()` (single-flight
+  per process, ~5 s timeout, failure → null = unknown) probes the same binary worker spawns
+  use, plus hand-rolled `comparePiVersion` (no semver dependency — deliberate, see the module;
+  malformed → null, conservative). Non-hermetic parity spawns now append `--no-mcp` when the
+  host pi is ≥ 1.0.4 — disconnecting ambient MCP servers, strictly stronger than the U2-lite
+  `mcp__*` tool-cut (kept as the cross-version-safe baseline). Below the floor or unknown →
+  no flag; hermetic path unchanged.
+
+### Fixed
+
+- **RPC transport lifecycle hardening (DR1/DR5)** — the experimental RPC worker transport can
+  no longer hang a worker-cap slot forever: the settle wait races `client.exited()` + a
+  belt-and-suspenders turn timeout (env `PI_CREW_RPC_TURN_TIMEOUT_MS`, default 600 000 ms,
+  env-only while RPC stays experimental; invalid/≤0 → default). Exit-before-`agent_settled`
+  and turn-timeout map to the early-failure shape so the stdio fallback engages and the slot
+  is released. DR5 closes the double-execution window: a session event after the prompt marks
+  the turn as started (`result.rpcAgentStarted`), and started-but-unsettled failures are
+  surfaced, never retried. New seam tests: crash-before-settle (fallback + slot released),
+  started-then-fail (no fallback), turn timeout (early-failure shape).
+- **`consumeAnsi` handles APC sequences (U6/DR4)** — an APC branch (ESC `_` … terminated by
+  BEL 0x07 or ST ESC `\`) sits beside the OSC branch with mirrored terminators and
+  control-char fallback, so pi-tui's `CURSOR_MARKER` (an APC string) measures zero-width;
+  `truncateToWidth`/`wrapHard` no longer count it as ~6 visible columns.
+
+- **herdr liveness ping survives ESM hosts** (battery 2026-10-07, T10c) — `pingSocketSync`'s eval'd
+  Worker used bare `require(...)`; under a `--input-type=module` host the eval source parses as ESM,
+  `require` throws, and the ping fail-closed to false → herdr degraded "socket not live" against a
+  LIVE socket (CJS hosts like the pi CLI were unaffected). The worker source is now dual-mode
+  (`require` try → dynamic `import()` fallback) and exported for tests;
+  `test/unit/runtime/surface/resolve-surface-ping.test.ts` pins both module contexts + fail-closed.
+- **resume-liveness F1 test stabilized** (battery 2026-10-07) — the "re-checks liveness INSIDE the run lock"
+  test raced the lock ACQUISITION itself (test vs resume; ~1/3 red even isolated — it blocked the new
+  pre-push hook twice). Entered-signal now guarantees the lock is held before the resume starts, making
+  both orderings of the pre-lock check deterministic refusals. 8/8 green isolated (was ~2/3).
+
+### Ops notes
+
+- **Azure provider rename in pi 1.0.3 (U3)** — pi renamed provider id
+  `azure-openai-responses`→`azure`; `AZURE_OPENAI_*` env vars are unchanged. User config
+  (`auth.json`/`models.json`/`settings.json`) entries under the old id need renaming.
+  pi-crew code is unaffected — `env-filter.ts` PROVIDER_ENV_KEY_MAP already keys `azure`
+  and `azure-openai` (both → `AZURE_OPENAI_*`).
+
+
+## [0.11.7] — Pi 1.0.0 adoption waves + deep-learn execution (2026-10-04)
+### Added — Pi 1.0.0 adoption wave (see docs/reviews/pi-1.0.0-adoption-review-2026-10-03.md)
+
+- **Session-file recovery for crash-killed workers (P1-1)** — workers now spawn with
+  deterministic `--session-id`/`--session-dir` (files live under the run's artifacts, not
+  `~/.pi`). On signal death (`exitCode === null`) the settle path tail-replays the worker's
+  session JSONL (SDK `parseSessionEntries`/`migrateSessionEntries`, torn-tail tolerant, 5s
+  bounded) and surfaces the last complete assistant message as a new `"session"` result
+  source — ranked below live captures, above stdout/stderr noise — plus a provenance sidecar
+  `results/<task>.session-recovery.json`. Config `runtime.sessionRecovery` (default **on**),
+  env `PI_CREW_SESSION_RECOVERY`.
+- **Worker auto-compaction observability (P2-2)** — the injected prompt-runtime extension now
+  listens for the SDK `session_before_compact` event and emits `worker.session_before_compact`
+  (scalar payload) through the worker-events channel. Hook is infra-proven in `-p` mode but
+  not yet live-fired at threshold (docs + host-analog evidence).
+- **Experimental RPC transport prototype (P2-3)** — new `src/runtime/rpc/`: strict-LF JSONL
+  frame client on SDK protocol types, `extension_ui_request` drain policy (count+drop), dialog
+  auto-answer policy `cancel`|`block` (auto-confirm deliberately absent). Config
+  `runtime.workerTransport: 'stdio' (default) | 'rpc'` + env `PI_CREW_WORKER_TRANSPORT` /
+  `PI_CREW_RPC_DIALOG_ANSWER`. The seam returns a structured not-implemented result until the
+  live-fire integration phase — default path is byte-identical.
+
+### Changed
+
+- **Surface TUI spawns pin `--tui-mode regular` (P0-1)** — Pi 1.0.0 defaults the TUI to
+  fullscreen, which silently empties the scrollback capture path (`capture-pane -S`) that
+  `readScreen` always uses. The pin restores pre-1.0.0 behavior for surface panes.
+- `parseRuntimeConfig` now emits `runtime.sessionRecovery` and `runtime.workerTransport`
+  (F19-1 discipline — both keys were previously parse-dead).
+- Legacy `@mariozechner/pi-coding-agent` fork resolution now warns once per process (staged
+  deprecation; canonical `@earendil-works` path unaffected).
+
+### Added — deep-learn round 3 execution (see docs/reviews/pi-1.0.0-deep-learn-r3-2026-10-03.md)
+
+- **Compaction-surviving worker header (R3-1)** — the run-static worker context (protocol
+  block, mailbox contract, workspace structure) moved from the user message to the
+  `--append-system-prompt` channel (byte-preserving move; probe-verified coexistence with
+  AGENTS.md discovery). Auto-compaction summarizes the user-message span but never the system
+  prompt, so long runs keep exactly the context that must not be lost. Prompt breakdown now
+  reports `userPrompt`/`userAppend` channels separately.
+- **Worker provenance self-report (R3-2)** — the handoff template's new `### Provenance`
+  section asks workers to echo their `PI_SESSION_ID`/`PI_SESSION_FILE`/`PI_MODEL`/`PI_PROVIDER`/
+  `PI_REASONING_LEVEL` env — model provenance without log parsing.
+- **Hermetic worker spawns (D5/R3-19)** — headless workers now spawn with `--no-extensions`
+  (default on; off-switch `runtime.hermeticWorkers` / `PI_CREW_HERMETIC_WORKERS`). Kills the
+  ~1.37 s/spawn ambient extension stack and the ambient untrusted tool surface (8 direct-exposure
+  MCP tools). Explicit `-e` prompt-runtime injection and `--skill` flags unaffected; surface TUI
+  panes keep the ambient stack.
+- **Trust pin (D1/R3-3)** — worker spawns pin `--no-approve` unconditionally: worker trust is
+  machine-state-free (no ambient `~/.pi/agent/trust.json` dependence). Project trust does not
+  gate write capability and AGENTS.md discovery is unaffected (probe-verified); surface TUI
+  spawns keep the interactive prompt.
+
+### Fixed
+
+- **Session args per-flag idempotency (live battery find)** — partial builder forwarding
+  (manifest `sessionId` without `sessionDir`) made the both-or-nothing guard self-disarm the
+  full activation: `--session-id` went out but `--session-dir` never did, so worker session
+  files silently landed in the cwd-keyed default dir and crash recovery read the wrong place.
+  Each missing flag is now filled independently; verified end-to-end by killing a real worker
+  (SIGKILL mid-tool) and recovering its last complete assistant message from the per-worker
+  artifacts dir in ~1s.
+- **engines floor raised to Node `>=22.19.0`** — aligns pi-crew with the SDK's own declared
+  floor (previously `>=22.0.0` let Node 22.0–22.18 users run an SDK that declares itself
+  unsupported).
+
+### Migration notes
+
+- **Hermetic worker spawns are ON by default** (`runtime.hermeticWorkers`): headless workers
+  no longer load ambient user extensions/MCP —
+  set `PI_CREW_HERMETIC_WORKERS=0` or `runtime.hermeticWorkers: false` to restore the old
+  behavior if a worker genuinely needs ambient tooling.
+
+### Fixed / removed
+
+- Dead `persist: true` widget option removed (call site + compat strip + type alias).
+- `--provider` without `--model` now has an argv contract test across every builder path
+  (Pi 1.0.0 turned the silent ignore into an error).
+
+
+### Security
+
+- **fast-uri 3.1.2 → 3.1.8** (transitive of `ajv`, shipped runtime tree): patched
+  the 9 GHSA advisories (host-confusion / SSRF-via-URI-parsing family). pi-crew
+  only compiles its own static JSON schemas with ajv, so exploitability was low,
+  but the package ships to users, so the lockfile bump is user-facing hardening.
+- Known-accepted (dev-tree only, NOT shipped — `npm pack` carries 0 copies):
+  `brace-expansion@5.0.9` nested under `@earendil-works/pi-coding-agent` →
+  `minimatch@10.2.6` carries 3 high ReDoS/stack-exhaustion advisories. Kept at
+  5.0.9 because npm's resolver deterministically re-nests 5.0.9 for this scoped
+  position (overrides, cache-clean, clean-room regen, and lockfile surgery all
+  fail or regress on plain `npm install`; only `npm ci` would hold the pin).
+  CI runs `npm ci --no-audit` and has no `npm audit` gate; revisit when the
+  SDK bumps minimatch.
+
 ## [0.11.6] — SDD-3/4 upgrade waves, W-E/W-I, small-wins, Pi 1.0.0 compat, GH #61 fix (2026-10-03)
 
 ### fix(extensions): GH-061 — normalize directory `pi.extensions` entries to entry-file identity (child workers double-load)
@@ -5023,7 +5239,6 @@ the user sees "Goal failed: Background runner died unexpectedly" instead of
 an infinite "running" state.
 
 
-
 ### `goal` — autonomous goal loop (P0a + P0 + P1)
 
 - `team action='goal' config.subAction='start|status|pause|resume|stop|step|clear'`.