pi-crew 0.9.61 → 0.9.64

package/CHANGELOG.md

@@ -2,6 +2,105 @@
 
 > **Note:** `atomic-write-v2.ts` / `AtomicWriter` mentioned in historical entries below was consolidated into `atomic-write.ts` as of v0.9.42. This changelog is preserved as historical record — the migration was completed (the v2 class was never adopted; v1 won on simplicity + symlink-safety + link+unlink atomicity). See `docs/migration/atomic-write-v2-migration.md` for the decision rationale.
 
+## [0.9.64] — pi-rlm→pi-crew pattern transfer: worker scratchpad + crash-resume + cancellation + quick wins (2026-08-09)
+
+### Quick Wins (patterns 17/19/20/11 + spike CI)
+
+### Features
+- **Schema-driven docs (QW17)**: `agents/*.md` frontmatter `tools:` now matches the
+  enforced `ROLE_TOOL_CONFIGS` (4 drifts fixed); a sync-test pins the derivation;
+  `scripts/gen-role-tools-docs.mjs` renders `docs/role-tools.md` from the source.
+- **Retry-resume contract suite (QW19)**: pins `executeWithRetry` (`?`-glob, empty-
+  retryableErrors, maxAttempts:0, abort-during-sleep, default attemptId),
+  `FileCheckpointStore` (corrupt-file quarantine, list-skip, wrong-runId delete),
+  and exports `shouldRecoverTask` for direct testing.
+- **Failure-mode inventory (QW20)**: `docs/failure-mode-inventory.md` maps the 7
+  pi-rlm failure modes to pi-crew handlers (wedge gap closed by Phase 1
+  ping-before-execute; EPIPE/timeout interplay declared as gaps).
+- **Error-as-data contract (QW11)**: `evidenceStatusFor` + `attemptErrorFor`
+  extracted as pure functions from `runChildProcessTask`; contract test pins the
+  precedence (cancelled > failed > completed; E007 timedOut override; 429 gated).
+- **`test:spike` script**: wires the scratchpad spike tests into CI.
+
+
+### Phase 3 cancellation hardening (experimental)
+
+### Features
+
+- **Atomic snapshot write (D2')**: `engine.snapshotState` now writes via
+  `temp + rename` (same directory), eliminating the theoretical torn-write race
+  between the debounce timer and the F3 quit flush.
+- **Kill-and-restore verified**: the existing SIGTERM → pi print-mode dispose →
+  `session_shutdown` reason:"quit" → F3 flush chain already captures a fresh
+  snapshot on worker kill (no new handler required). Documented + pinned by a
+  gated real-worker test.
+- **EngineBusyError: deliberately skipped** — ping-before-execute already
+  detects a wedged guest; the FIFO execute queue makes a busy-reject redundant.
+
+
+### Phase 2 crash-resume (experimental)
+
+### Features
+
+- **Cross-attempt restore (crash-resume)**: a scratchpad worker attempt N+1 (retry
+  / crash-recovery re-queue / manual re-run) automatically revives the namespace
+  from the previous attempt's redacted snapshot artifact — via a single
+  spawn-time lookup (`findLatestScratchpadSnapshot`), no retry-loop / recovery
+  changes needed. Restores lazily on the first `execute` call (D7 lazy invariant
+  kept) with a one-line model-visible notice.
+  - Lookup: latest mtime (model-fallback index resets each retry round → number
+    is not write-order); tie-break lowest attempt = newest round.
+  - Security: read-time re-validation (containment + filename pattern + lstat +
+    size + mtime hint, D10); fail-open (D11) never breaks the worker; strict scan
+    (D12); base64 round-trip (D13); redacted secret → literal `"***"` (D4).
+- **Guest zombie backstop (SEC-1)**: the engine now overrides
+  `PI_CREW_PARENT_PID=worker pid` for the guest (D5), so an orphaned guest
+  (worker SIGKILL'd) is flagged by the zombie scanner — pure inheritance left
+  guests LIVE forever holding provider keys + broker token.
+- **Snapshot cap (D6)**: 4 MiB two-sided (write-side raw byteLength trim, read-
+  side file + per-var 256 KiB) bounds v8.deserialize amplification.
+
+
+### Phase 1 worker scratchpad (experimental)
+
+### Features
+
+- **Worker stateful scratchpad (experimental)**: opt-in persistent Bun-free JS
+  evaluator (`execute` tool) for workers, ported from the pi-rlm pattern.
+  State (variables, parsed data) compounds across `execute` calls within a task
+  attempt — intermediate results live in a namespace instead of being re-derived
+  from transcript text. Snapshot per-attempt into the run artifact store
+  (redacted + atomic) prepares the ground for Phase 2 crash-resume.
+  - Opt-in per role: `executor`, `test-engineer`, `verifier` (default on); other
+    roles via agent frontmatter `scratchpad: true` (write roles only).
+  - Security: S-6 read-only roles are gated out regardless of frontmatter
+    (privilege-elevation guard); F6 `scratchpad: false` is an explicit kill-switch;
+    raw snapshots never land in the artifact root (temp dir → redacted
+    `writeArtifact`).
+  - Dormant by default: only active when `PI_CREW_SCRATCHPAD=1` is set by the
+    spawner; zero behavior change for non-opt-in workers.
+
+
+## [0.9.63] — built-in performance observability + local-path provider-extension discovery (2026-08-08)
+
+### Features
+
+- **Built-in performance observability (byte-built-in, always-on).** Every team run now auto-attaches a detached resource sampler and auto-generates a detailed performance report on completion — no separate benchmark harness needed. Toggle per-team via frontmatter `observability: true|false` (default `true` for parsed team files; direct-object `TeamConfig` fixtures stay unset for test isolation).
+  - **Live resource sampler** (`scripts/resource-sampler.mjs`): samples CPU/RSS per-PID every 2s via ppid-tree attribution (root runner + all child workers, including respawns), with PID-reuse guard (`/proc` starttime), first-sample CPU exclusion, and **6 live warning categories** — `high_cpu` (≥300%), `rss_jump` (+200MB/interval), `rss_high` (≥1GB), `rss_leak` (window-30 monotonic +100MB), `proc_died`, `proc_zombie`. Rate-limited (10s/pid/category); `--no-live-warn` flag to silence.
+  - **Post-hoc analyzer** (`scripts/analyze-run.mjs`): combines `events.jsonl` + transcripts + `resources.jsonl` into a markdown report at `docs/perf-report-<runId>.md` with **22 anomaly categories** (task_failed, model_retry/model_cascade, large_gap, slow_phase, launch_delay, drain_stall, token_imbalance, no_cache, worker_respawn_churn, api_error_storm, zero_output_completion, sustained_cpu, transient_cpu/rss_spike, rss_growth, run_not_completed, high_failure_rate, run_idle, cost_unreported, missing_transcript, sampler_gap, tool_churn), a per-subagent timeline (launch / respawn / startup / active-work / drain / finalize), and token/cost/model attribution. Optional `--agents` flag emits per-agent breakdowns.
+  - **Runtime wiring** (`src/runtime/team-runner.ts`): `startPerfSampler` spawns the sampler detached + `unref`'d (death never affects the run); `schedulePerfAnalyze` runs the analyzer +3s after `after_run_complete` via an `unref`'d `setTimeout` (never blocks run completion). Strict `observability !== true` keeps test fixtures from spawning. The sampler auto-stops when the run manifest reaches a terminal status (`--watch-run` mode).
+  - **Overhead ≈ 0** (measured A/B): sampler ~0.05% of one core, 56MB RSS fixed; analyzer ~72ms one-shot after run; ~32KB artifacts/run. Verified end-to-end on real runs — see `docs/real-test/reports/real-test-2026-08-07-perf-obs-overhead.md`.
+
+### Bug fixes
+
+- **Local-path provider extensions were not discovered for child workers (oc-go and any `pi install <local-path>` provider went "Model not found").** `discoverProviderExtensions` only resolved `npm:` specs from `~/.pi/agent/settings.json` `packages`, skipping local-path specs (e.g. `../../source/my_pi/source/pi-other-provider`) on the assumption that local paths were the pi-crew extension itself. That assumption was wrong for local provider extensions: `--no-extensions` in `buildPiWorkerArgs` stripped the provider from every child spawn → every model from that provider hit `Error: Model "…" not found` → the fallback chain burned 5 spawn-fails (~10s) per task before landing on a builtin provider. Fix: resolve local-path specs (`./`, `../`, absolute) relative to the settings.json dir — same trust level as `npm:` (settings packages are a sanctioned channel written by `pi install`); skip the pi-crew package itself via `packageRoot()` (a worker must not re-load the orchestrator). **SEC-1 preserved** (project/project-pi AGENT extensions in `.crew/agents/*.md` frontmatter stay gated — that is a separate, untrusted channel). Tests in `test/unit/runtime/model/provider-extensions.test.ts`. Investigation + correction of the earlier "hidden models" mis-attribution in `docs/real-test/reports/real-test-2026-08-08-provider-ext-local-path.md`.
+
+## [0.9.62] — provider-quota attribution per live-session agent + dead-worker alert re-fire fix (2026-08-06)
+
+### Bug fixes
+
+- **Provider-quota responses were attributed to the wrong provider in the opt-in `live-session` runtime.** pi's `after_provider_response` event carries no provider/sessionId, so quota was keyed off the shared module-scoped `currentSessionModel()`: a provider-B response (e.g. a 429) was recorded under the main session's anchor provider A — mis-deprioritizing A, or overwriting a genuine exhaustion signal so the “deprioritize exhausted providers” contract was not reliably honored. (Only reachable via the opt-in `live-session` runtime; the default `child-process` runtime is unaffected since each worker is its own process with anchor==serving provider.) Fix: carry each in-process live agent's resolved model through `AsyncLocalStorage` and attribute quota from it (`resolveProviderForResponse()`, `src/runtime/model/session-model.ts`); **skip** attribution (rather than guess) when live agents are active but the async context is absent; the default `child-process` path is byte-identical to before. Hardening from review: `unregisterLiveAgentModel` now runs **first** in the live-session `finally` block (a thrown `terminateLiveAgent` on the user-cancel path previously skipped it, permanently blinding quota process-wide via `hasActiveLiveAgents()` stuck `true`); `liveAgentModels` is capped at 5,000 entries matching `MAX_LIVE_AGENTS`. Tests in `test/unit/runtime/model/live-agent-quota-attribution.test.ts`. Investigation/spec in `docs/bugs/model-quota-attribution.md`.
+- **Dead-worker dashboard alert re-fired every 5 min for already-terminal runs.** A stale snapshot cache (not auto-invalidated when a run transitions to terminal) kept task status as "running" with an aged heartbeat, so the health loop re-classified it as dead and re-emitted the alert every 5 min; and there was no dismiss/clear path for health notifications. Fix: (1) the health loop in `renderTick` now re-verifies each run against a FRESH manifest-cache read before emitting; terminal runs are skipped and their stale snapshot is purged; (2) `NotificationDescriptor` gains an optional `clear` flag — when a run goes terminal, previously-emitted `recovery_dead_workers` / `recovery_missing_heartbeat` notifications are cleared and their cooldown entries are removed. Genuinely running runs with stale workers still alert (regression-guarded). Tests in `notification-router.test.ts` and `heartbeat-aggregator.test.ts`. Investigation in `docs/bugs/dead-worker-alert-refire.md`.
 
 ## [0.9.61] — bundle republish: bug-44 fix shipped in dist (2026-08-05)
 

package/README.md

@@ -69,6 +69,7 @@ repo: https://github.com/baphuongna/pi-crew
 - **Durable event replay** (L1, v0.9.8) — `RunEventBus.onWithReplay()` catches up a re-subscribing dashboard/overlay with events it missed during transient absence (toggle, reconnect), replaying from the durable JSONL log with seq-based dedup. No information loss even if the live subscriber was briefly gone.
 - **Lossless-by-default output handling** (L4, v0.9.8) — worker output thresholds sized from measured data (100% of real outputs fit without compaction); when compaction is unavoidable it keeps head+tail (preserves closing code fences/headings) instead of head-only truncation. No more `[pi-crew compacted N chars]` markers eating the end of a worker's result.
 - **Inter-pi broker** (v0.9.47, default-on) — a Unix-domain-socket message bus that lets concurrently-running Pi sessions pass messages, steering notes, and task-status events to each other. **On by default** on Linux + macOS; auto-disabled on native Windows (no unix socket). Three independent kill switches: `broker.enabled: false` (config), `PI_CREW_BROKER=0` (env, always wins), Windows auto-disable. See [docs/decisions/2026-07-22-broker-phase4-gated-on.md](docs/decisions/2026-07-22-broker-phase4-gated-on.md).
+- **Worker stateful scratchpad** (experimental, opt-in per role) — `executor` / `test-engineer` / `verifier` workers get a `scratchpad` tool: a persistent Bun-free JS evaluator whose namespace **compounds across calls within a task attempt** (variables set in one cell are visible in the next), so intermediate results live in memory instead of being re-derived from the transcript. Snapshots are flushed (redacted, atomic) per-attempt into the artifact store; the next attempt (retry / crash-recovery re-queue / re-run) **automatically revives the namespace** from the latest snapshot. Dormant by default (armed only when the spawner sets `PI_CREW_SCRATCHPAD=1`); zero behavior change for non-opt-in workers. Ported from the `@shift-labs/pi-rlm` pattern. See [src/runtime/scratchpad/README.md](src/runtime/scratchpad/README.md) (Phase 1-3 design, env keys, guards, threat model).
 - **`test:critical` + `real-test-pi-crew` skill** (v0.9.47) — a curated 14-file / 97-test subset (`npm run test:critical`, ~20s) for fast in-loop verification, plus a bundled skill distilling the full 8-tier end-to-end verification discipline (unit → 3-path kill-switch proof → typecheck/bundle → live TUI probing → smoke team run). Prevents the verifier-worker hang that full `npm test` (>4 min) caused against the 300s worker timeout.
 - **Provider extensions in subagents** (v0.9.57) — pi-crew spawns child-pi workers with `--no-extensions` (security posture), which made extension-registered providers (e.g. `pi-commandcode-provider`) unresolvable inside subagents. pi-crew now **auto-discovers provider packages** from `~/.pi/agent/settings.json` `packages` (npm: specs) and loads them via `--extension` in every builtin/user subagent — so **all provider models work in subagents**. An explicit `runtime.agentExtensions: string[]` config is an optional extra allowlist on top of auto-discovery. **SEC-1 preserved:** project/project-pi agents never receive these (env-gate unchanged).
 

package/agents/critic.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls
+tools: read, grep, find, ls, glob
 ---
 
 You are a critical reviewer. Find flaws, missing steps, unsafe assumptions, overengineering, underengineering, and verification gaps. Return concrete fixes to the plan.

package/agents/explorer.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls
+tools: read, grep, find, ls, glob, bash
 ---
 
 You are a fast codebase explorer. Map relevant files, symbols, data flow, and constraints. Do not modify files. Return concise findings with paths and evidence.

package/agents/planner.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls
+tools: read, grep, find, ls, glob
 ---
 
 You are a planning specialist. Convert the goal and discovery notes into a concrete, ordered plan. Identify dependencies, risks, validation steps, and handoff instructions for implementers.

package/agents/reviewer.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls, bash
+tools: read, grep, find, ls, glob, bash
 ---
 
 You are a code reviewer. Review the implementation for bugs, regressions, maintainability issues, missing tests, and project-rule violations. Return prioritized findings with evidence.

package/agents/security-reviewer.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls, bash
+tools: read, grep, find
 ---
 
 You are a security reviewer. Look for injection, authn/authz flaws, insecure defaults, secret exposure, unsafe filesystem/network behavior, and dependency risks. Return severity and remediation.

package/agents/test-engineer.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls, bash, edit, write
+tools: read, edit, write, bash, ls
 ---
 
 You are a test engineer. Identify the right test level, add or adjust tests when asked, detect flaky assumptions, and report exact validation commands and results.

package/agents/writer.md

@@ -5,7 +5,7 @@ model: false
 systemPromptMode: replace
 inheritProjectContext: true
 inheritSkills: false
-tools: read, grep, find, ls, edit, write
+tools: read, edit, write, ls
 ---
 
 You are a documentation specialist. Produce clear, concise, maintainable docs and summaries. Preserve technical accuracy and avoid marketing fluff.