pi-crew 0.9.33 → 0.9.35

package/CHANGELOG.md

@@ -1,5 +1,181 @@
 # Changelog
 
+## [0.9.35] — performance plan Phases 1–3 + 2 wiring fixes (2026-07-13)
+
+Shipped the **forward performance optimization plan** in 3 phases (`docs/performance-optimization-execution-plan.md`) plus 2 follow-up wiring fixes for the `pipeline` workflow and `schedule` action. 8 performance optimizations landed + 1 bundle rebuild. End-to-end verified post-restart via `team action='run', workflow='pipeline', team='research'` (`team_20260713102348_529c8e4e79e90e20`, 4/4 tasks, 142,686 tokens, 6.4 min, pipeline-summary.md emitted with "PIPELINE_WORKFLOW_OK" + scheduled job successfully removed via new subAction). Typecheck clean, 63 + 58 + 92 targeted unit tests pass across the modified suites.
+
+**Important deployment note** (CRITICAL for anyone updating pi-crew from v0.9.34 or earlier): the extension loads `dist/index.mjs` (a pre-built bundle) by default since v0.9.17. Source edits to `src/` do NOT take effect until `npm run build:bundle` runs and the Pi extension cold-starts. The bundle rebuild (`a15c1a8`) is part of this release. See "Bundle resolution order" below.
+
+### Phase 1 — Quick Wins (commit `e316a36`)
+
+Four LOW-risk optimizations that eliminate synchronous I/O on critical paths:
+
+- **OPT-02 — `saveRunManifest` async migration**: 14/16 sync call sites converted to `await saveRunManifestAsync()` across 8 files (team-runner.ts, task-runner.ts, background-runner.ts, adaptive-plan.ts, team-tool.ts, team-tool/goal-wrap.ts, team-tool/api.ts, team-tool/goal.ts). The 2 remaining sync calls in `stale-reconciler.ts` are documented as intentional (their sync call chain via `crash-recovery.ts: withRunLockSync` and `widget-model.ts` cannot be safely migrated). Special case: `task-runner.ts` swap from `withRunLockSync` to `await withRunLock`.
+- **OPT-03 — Guard `collectedJsonEvents` allocation in `live-session-runtime.ts`**: Conditional allocation based on `yieldEnabled` flag. Avoids ~1KB array allocation per task when yield detection is disabled. Pattern mirrors `task-runner.ts:165`. Unified the flag with the late yield-detection block to prevent a latent mismatch when `DEFAULT_YIELD_CONFIG.enabled=false`.
+- **OPT-05 — Skill cache stats exposed**: `getManifestCacheStats()` returns `{ size, maxEntries, ttlMs, perStateRoot: { generation, ageMs } }`. Surfaced via `team action='cache'`. Documented why `SKILL_CACHE_MAX_ENTRIES = 128` is well-tuned (4× headroom over 30 package skills).
+- **OPT-06 — Async transcript writes**: Fire-and-forget `appendTranscriptAsync` (replaces `mkdirSync`/`appendFileSync` in `child-pi.ts:413`, `live-session-runtime.ts:152`). Also converted steering writes and background-log writes in `task-runner.ts`. Best-effort writes; loss acceptable.
+
+**Files changed (Phase 1)**: 13 files, +387/-77 lines.
+
+### Phase 2 — Independent Optimizations
+
+Two MEDIUM-effort changes with no shared dependencies:
+
+- **OPT-04 — Code-aware token estimation** (commit `3f484af`): `countTokens(text: string)` now detects code-heavy content via bracket/semicolon/colon density + multi-char operator scoring (threshold 0.10) and applies a code-specific formula `max(ceil(alpha/3.5), alphaRuns) + punct - multiCharOps`. Prose path unchanged. Single-pass O(n) preserved. Tool-output-pruner verified unchanged behavior.
+  - **Files**: `src/utils/token-counter.ts`, `test/unit/token-counter.test.ts` (+341/-36).
+- **OPT-01 — Streaming dispatch** (commit `4ffd954`): Replaced batch-level `mapConcurrent` + `await all` pattern in `executeTeamRunCore` with a streaming dispatch loop. Uses `pendingUnits` Map + `Promise.race` to dispatch new tasks as soon as a concurrency slot frees (instead of waiting for the entire batch to complete). All existing semantics preserved: lock-based merges (`mergeTaskUpdatesPreservingTerminal` under `withRunLock`), retry, cancellation, hooks, coalescing, phase advancement, budget enforcement, concurrency limits (workflow defaults, hard cap 8), DAG readiness recomputation after each task completion.
+  - **Files**: `src/runtime/team-runner.ts`, `test/unit/team-runner-merge.test.ts` (+231/-39).
+  - **Expected impact**: 20-40% latency reduction for workflows with mixed task durations.
+
+### Phase 3 — Structural Changes
+
+Two architecturally significant changes:
+
+- **OPT-08 — In-memory state helpers** (commit `5957cc6`): Added `unloadRun(stateRoot: string): Promise<void>` (flushes pending coalesced atomic writes + invalidates the cache entry) and `getManifestCacheStats()` accessor. **Scope reduced from full in-memory authoritative state refactor** because:
+  - Existing `manifestCache` already has TTL/LRU/generation counter/mtime invalidation
+  - `persistSingleTaskUpdate`'s 100-iteration CAS loop is load-bearing for async-notifier, crash-recovery, and parallel-batch writer topologies
+  - Sync callers in `stale-reconciler.ts:78-79` and `crash-recovery.ts` cannot be migrated safely
+  - Pin/unpin skipped (intentional): TTL is rarely an issue during active runs because every save re-stamps `cachedAt`
+  - Public API of `saveRunManifest`/`saveRunTasks`/`loadRunManifestById` unchanged. The full in-memory refactor (5-8 day effort, VERY HIGH risk) is recommended as separate future work.
+  - **Files**: `src/state/state-store.ts`, `test/unit/state-store.test.ts` (+256/-1). 4 new tests covering: cache cold start, mutation reflection, post-unload disk read, concurrent writers via existing CAS.
+- **OPT-07 — Live-session migration** (NO code change required): The live-session runtime already slots in cleanly to the OPT-01 streaming dispatch via the `runtimeKind`-agnostic `runTeamTask` envelope. 23 behavioral invariants verified preserved without edits.
+
+### Feature fixes (separate from the perf plan)
+
+- **Pipeline workflow wiring** (commit `9febb6e`): `pipeline` workflow previously returned "is not yet wired into the team execution system" when invoked. Removed the early-return block in `src/extension/team-tool/run.ts` so pipeline now flows through the standard `validateWorkflowForTeam` → `executeTeamRunCore` path (same as `research`/`default`/`implementation`). Restructured `workflows/pipeline.workflow.md`: step IDs shortened (`Stage 1: Research` → `research` to avoid parser edge cases with colons in step IDs), `dependsOn` updated, `output: pipeline-summary.md` added to final stage, final stage writes `PIPELINE_WORKFLOW_OK` for verification.
+  - **Verified post-restart** (Round 7): `team action='run', team='research', workflow='pipeline', goal='...'` ran all 4 stages end-to-end (`team_20260713102348_529c8e4e79e90e20`, 142,686 tokens, 382,330ms, pipeline-summary.md created).
+- **Schedule sub-actions** (commit `0c653a1`): Discovered gap — scheduled jobs had no cancellation path through the team tool. A `cron='* * * * *'` job would fire every minute forever. Added `handleRemoveScheduled` and `handleUpdateScheduled`. Route via `subAction='remove'|'delete'|'disable'|'enable'|'update'` with `jobId` arg.
+  - **Verified post-restart** (Round 7): `team action='schedule' subAction='remove' jobId='6768c754-...'` returned `"Scheduled job removed."`.
+  - **Usage**:
+    ```
+    team action='schedule' subAction='remove'  jobId='<uuid>'
+    team action='schedule' subAction='disable' jobId='<uuid>'
+    team action='schedule' subAction='enable'  jobId='<uuid>'
+    team action='schedule' subAction='update'  jobId='<uuid>' cron='0 9 * * MON'
+    ```
+  - **Files**: `src/extension/team-tool/handle-schedule.ts` (+101 lines), `src/schema/team-tool-schema.ts` (+4 lines typed `subAction?` + `jobId?`), `test/unit/team-tool-schedule.test.ts` (+144 lines, 5 new tests, 16/16 pass).
+
+### Regression fixes (caught by CI)
+
+Three regressions from the performance optimizations were caught by CI's full test matrix and fixed before publish:
+
+- **Transcript flush race** (commit `4f3b005`): OPT-06 (async transcripts) broke 3 integration tests that read the transcript file immediately after `runChildPi` returns — the async file handle had not yet been opened or flushed, so tests saw ENOENT or empty content. Fix: module-scoped `Set<Promise<void>>` tracks in-flight writes; `flushPendingTranscriptWrites()` drains them before lifecycle boundaries (`ChildPiLineObserver.flush()` and `runChildPi` settle). Also: `tailReadWithLineSnap` falls back to `fallbackContent` when the file exists but is empty (fire-and-forget file may exist before content flushes). This also fixed `implementation-fanout.test.ts` which relied on the adaptive mock's transcript.
+- **Atomic-write EEXIST + ENOENT stat races** (commit `e59b129`): OPT-02 (async `saveRunManifest`) introduced two interleaving races in `renameWithLinkAsync` (unlink+link as separate async syscalls):
+  1. Two concurrent saves race on `link(tmp, dest)` — the second hits EEXIST. Fix: add EEXIST to `RETRYABLE_LINK_CODES`; the retry loop unlinks+re-links.
+  2. After EEXIST retry, the retry's `unlink(dest)` removes the file between another save's `atomicWriteJsonAsync` and its `stat(dest)` cache update. Fix: catch ENOENT on stat, use fallback `{0,0}` values — cache re-reads from disk on next load.
+  - This fixed the `parallel-research-dynamic.test.ts` macOS/Ubuntu CI flake.
+- **Import sort + format** (commits `538138e`, `3e43f3b`): biome lint/format CI checks.
+
+### Bundle resolution order — read this before updating from v0.9.34 or earlier
+
+This release fixes a silent cache-staleness trap. The `index.ts` entry point resolves the extension load in this order:
+
+1. `dist/index.mjs` (pre-built bundle) — DEFAULT since v0.9.17. Built by `scripts/build-bundle.mjs` from `index.bundle.ts`.
+2. Inline strip-types loading — fallback when bundle missing OR `PI_CREW_USE_BUNDLE=0`.
+
+**Implication for users updating from a v0.9.17+ version**: source edits in `src/` are NOT immediately visible to a running Pi session. The bundle must be rebuilt (`npm run build:bundle`) and the Pi extension must cold-start before source changes take effect. `npx pi install .` does NOT auto-rebuild the bundle (postinstall is best-effort and may silently skip). This release ships `commit a15c1a8` which rebuilds the bundle to reflect all 8 source-level commits.
+
+**To force strip-types loading** (always uses fresh source): set env var `PI_CREW_USE_BUNDLE=0` at extension startup. Cannot be changed mid-session.
+
+### Verification
+
+- **Typecheck**: `npx tsc --noEmit` clean across all 8 commits.
+- **Targeted unit tests**: 63 (Phase 1) + 58 (Phase 2) + 92 (Phase 3) = **213 tests pass** across the modified suites.
+- **End-to-end (post-restart)**:
+  - `team_20260713102348_529c8e4e79e90e20`: 4/4 pipeline stages, 142,686 tokens, 6.4 min, pipeline-summary.md written.
+  - `team action='schedule' subAction='remove' jobId='6768c754-...'` → "Scheduled job removed."
+- **Bundle**: `grep -c "is not yet wired" dist/index.mjs` → 0; `grep -c "handleRemoveScheduled" dist/index.mjs` → 4 (was 0).
+- **Backward compat**: all existing workflows (`default`, `implementation`, `research`, `fast-fix`, `parallel-research`, `chain`, `plan-execute`, `review`) unchanged.
+
+### Commits
+
+```
+a15c1a8 chore(release): rebuild dist bundle (0.9.34 + perf+feat commits)
+0c653a1 feat(schedule): expose remove/disable/enable sub-actions
+9febb6e feat(workflows): wire pipeline workflow into execution
+5957cc6 perf(state-store): additive cache helpers (OPT-08 minimal)
+4ffd954 perf(team-runner): streaming dispatch (OPT-01)
+3f484af perf(token-counter): code-aware estimation (OPT-04)
+e316a36 perf(runtime): Phase 1 performance optimizations
+c9f03ac docs(perf): add performance optimization execution plan
+```
+
+### Lessons distilled
+
+1. **Two-layer indirection is a footgun**: symlink IS live for source files but the extension loads a pre-built bundle. Always rebuild bundle after committing source changes.
+2. **OPT-08 scoping matters**: the temptation to do a full rewrite was strong, but the safety review revealed multiple "defer trigger" conditions that justified additive helpers only. The 8-day rewrite is now a separate, gated future project.
+3. **API gap discovery**: the `schedule` cron pitfall (no cancel path) was found by accident during E2E testing. Treating the discovery as a fix opportunity instead of a side note yielded `0c653a1` — a clean API addition that closes the gap.
+4. **User challenges matter**: the "is this cache?" question surfaced the actual root cause (stale bundle). My initial "ESM cache" hypothesis was partially wrong; the user pushing back forced me to look at `dist/index.mjs` and find the real culprit.
+5. **Pipeline workflow `Stage 1: Research` → `research`**: colons in step IDs caused parser edge cases in `parseWorkflowFile`. Always use clean identifier-form IDs in workflow steps.
+6. **`PI_CREW_USE_BUNDLE=0`** is the escape hatch when bundle is stale and you can't rebuild immediately. Slower startup (~19% per `scripts/bench-cold-start.mjs`) but always fresh source.
+
+## [0.9.34] — performance + security audit quick wins (2026-07-11)
+
+Shipped the **forward performance audit** follow-up: 5 batches of perf + security fixes derived from the forward audit (`team_20260711095156_8a8a56a3c7dff269`) and the bundled code + security review. 15 audit findings + 1 security fix all closed. End-to-end verified on a live `team` run (`team_20260711134929_498a39322bc756d9`, 3/3 tasks, 148k tokens, real `tasks.json`/`events.jsonl`/`heartbeat.json` produced). Typecheck clean, lint clean, biome format clean, 146/146 targeted tests pass, full unit suite cut at 2,894 with **0 failures** (test runner timed out only on long backoff tests).
+
+Detailed methodology: [`docs/perf/performance-audit-report-2026-07.md`](docs/perf/performance-audit-report-2026-07.md) · commit history below · end-to-end validation: [`/tmp/perf-audit-validation.md`](/tmp/perf-audit-validation.md).
+
+### Performance
+
+**Phase 2 — HIGH severity (P1, P2, P3)** + LOW (P10) + SEC-M1 (committed `f8e5629`):
+- **P1** — `withQueue()` returns the same task reference when the computed queue matches `task.graph.queue`. Allocation-free on the common case; reference-stable for downstream selectors.
+- **P2** — `saveRunManifest` / `saveRunManifestAsync` capture `manifestCache` before invalidating so the cached tasks array is reused on write-through repopulation. Eliminates per-call `JSON.parse(readFileSync(tasks.json))`.
+- **P3** — `persistSingleTaskUpdate` reuses caller-supplied `fallbackTasks` on the first CAS attempt; only re-reads disk on actual contention.
+- **P10** — `mergeTaskUpdatesPreservingTerminal` replaced its `O(N*M)` `.find()` + `.map()` nested loops with a single-pass `Map<id, task>` index (1000 → 120 ops).
+- **SEC-M1** — `validateEndpoint` blocks `fe80::/10` (link-local), `fec0::/10` (site-local), `ff00::/8` (multicast), `::` (unspecified), and `::ffff:x.x.x.x` (IPv4-mapped) in addition to `::1` and `fd`/`fc`.
+
+**Phase 3 — MEDIUM (P5, P7)** (committed `1e8d5b2`):
+- **P5** — `isTargetNotSymlink()` cached for 1s on a per-file basis; `invalidateSymlinkSafeCache(dir)` flushes both cache layers.
+- **P7** — 5 fire-and-forget call sites in `live-session-runtime.ts` migrated from sync `appendEvent` (with `sleepSync` lock) to `appendEventFireAndForget`. SIGTERM/AbortSignal handlers now fire promptly during high-frequency prompt-done churn.
+
+**Phase 4 — MEDIUM-LOW (P6, P9)** (committed `824f428`):
+- **P6** — `writeProgress()` replaced its O(N²) `findIndex` dedup with a single-pass `Map<path, ArtifactDescriptor>`. Added a content-skip cache (WeakMap on manifest) for back-to-back `applyPolicy` + `executeTeamRun` calls within the same millisecond.
+- **P9** — `computeStablePrefixComponents()` gained a 60s TTL cross-run cache keyed on `(cwd, step.task)`. Sequential runs in the same session skip the 3 awaits (`buildWorkspaceTree` + `runRetrievalCycle` + `buildKnowledgeFragment`). `clearStablePrefixCache()` clears both cache layers.
+
+**Phase 5 — remaining LOW tier (P11, P12, P13)** + P14 / P15 closure (committed `121cc55`):
+- **P11 + P14** — `buildTaskGraphIndex()` identity-memoized via WeakMap. Back-to-back calls within one main-loop iteration share the 3 cached structures (`doneSteps` Set, `idMap` Map, `stepToTaskId` Map).
+- **P12** — `mapConcurrent()` short-circuits on `length === 0` and `length === 1`.
+- **P13** — Team-runner heartbeat interval: 30s → 60s. Stale threshold is 5min (5 ticks of slack).
+
+**Biome auto-format** (committed `c329cc7`): organized imports + line-width reformat for the import introduced in P6 (`hashArtifactContent as hashContent` + `writeArtifact` alphabetized into existing group).
+
+### Verification
+
+- **Typecheck**: `tsc --noEmit` clean (including strip-types `import('./index.ts')`).
+- **Lint**: `biome check --linter-enabled=true` clean (1125 files, 0 errors).
+- **Format**: `biome format --check .` clean.
+- **Conflict markers / lazy imports**: clean.
+- **Targeted unit tests**: **146/146 pass** across 20 test files directly touched by these changes:
+  - `atomic-write-symlink`, `atomic-write-edge-cases`, `atomic-write`, `artifact-store`
+  - `event-log`, `event-log-rotation`, `event-log-async-queue`
+  - `live-session-import-latch`, `live-session-context`, `live-session-health`, `live-session-health-cov`
+  - `parallel-utils`, `team-runner-merge`, `team-runner-heartbeat`, `task-runner-heartbeat`
+  - `heartbeat-watcher`, `heartbeat-aggregator`, `heartbeat-gradient`
+  - `prompt-builder-cov`, `task-graph-scheduler`
+  - `otlp-ssrf`
+- **Full unit suite**: 2,894 pass with **0 failures** at the time-of-test cutoff (test runner timed out only on retry/backoff tests with 1–15s waits).
+- **End-to-end**: a real `team action='run' workflow='research' goal='extract P1/P2/P3/SEC-M1 ...' ` (`team_20260711134929_498a39322bc756d9`) completed end-to-end (3/3 tasks, 148k tokens, 5.5 minutes) with valid artifacts:
+  - `manifest.json` 23kB (P2 cached write-through stamp exercised)
+  - `tasks.json` 73kB (P3 first-attempt reuse exercised)
+  - `events.jsonl` 50kB, 132 events (P7 async fire-and-forget exercised)
+  - `heartbeat.json` valid (P13 60s interval exercised)
+  - `progress.md` (P6 dedup exercised)
+- **Validation report**: `/tmp/perf-audit-validation.md` (9.2kB) verifies each fix site at file:line against current HEAD `121cc55`.
+
+### Tests Added (12 new)
+
+- `atomic-write-symlink.test.ts`: 2 P5 cache invariants
+- `parallel-utils.test.ts`: 2 P12 single-item + empty-array fast paths
+- `task-graph-scheduler.test.ts`: 2 P14 cache-by-reference + invalidation
+- `prompt-builder-cov.test.ts`: 2 P9 cross-run cache + invalidation
+- `otlp-ssrf.test.ts`: 5 SEC-M1 IPv6 prefix tests (fe80::/10, fec0::/10, ff00::/8, ::, ::ffff:)
+
+### Honest framing
+
+- **False-positive catch in production**: the live-validation report (`/tmp/perf-audit-validation.md`) flagged P2 as "⚠️ CLOSED-PARTIAL — write-through cache stamp not located in scanned range". Manual verification at `state-store.ts:332-371` confirms the fix IS in place and the analyst's scanned range (`466-507`) was just outside the actual fix site. Documented in the validation report.
+- **Exceeds spec**: SEC-M1 implementation blocks more IPv6 reserved prefixes than the audit requested (audit cited `fe80:` + `fec0:`; implementation also adds `ff00:`, `::`, `::ffff:`). The audit-defined minimum bar is met; the additional coverage is documented in the diff and the comment block in `otlp-exporter.ts`.
+- **Backstop analysis**: P2's "invalidate-before-write" is intentional (preserved across the fix). The crash-safety invariant is documented in the inline comment; performance claim is "removed the redundant post-write read of tasks.json", not "removed invalidation".
+
 ## [0.9.33] — correctness hardening + reverse audit (2026-07-11)
 
 Shipped the **forward performance audit** quick wins, then ran a **reverse audit** (6 parallel bug-hunters → firsthand verification) that surfaced 9 real correctness bugs the forward audit missed. **10 confirmed bugs fixed** total, plus a quality fix and a durable regression bench. End-to-end verified with a real `team` run in this session (all fixes exercised on a live workload).
@@ -29,6 +205,41 @@ The reverse audit ran 6 parallel bug-hunters across concurrency, state machine,
 
 - **NEW-C3** — terminal manifest+tasks persistence race. `runTeamTask` wrote `manifest.json` (immediate) and `tasks.json` (via `persistSingleTaskUpdate` → `saveRunTasksCoalesced`, 50 ms debounce) **outside any lock**, racing the team-runner batch merge which writes both under `withRunLock`. A worker could persist its terminal artifacts → the merge's `loadRunManifestById` reads the worker-written manifest → merge overwrites with the worker's stale `tasks` (which lacks the merge's attempt enrichment + graph recompute) → silent data loss. Fix: wrap `saveRunManifest` + `persistSingleTaskUpdate` in a single `withRunLockSync` so the whole terminal state lands atomically. (`a03d244`)
 
+### Performance — forward audit quick wins (Phase 5 — remaining LOW-tier P11/P12/P13/P14)
+
+Closing out the LOW-tier items from the audit. The HIGH and MEDIUM-tier items (Phase 2-4) are already shipped.
+
+- **P11 + P14 (LOW)** — `buildTaskGraphIndex` (and transitively all callers through `ensureIndex`) now memoizes by array reference via a WeakMap. Back-to-back calls with the same tasks array (common within a single main-loop iteration) reuse the 3 cached data structures (`doneSteps` Set, `idMap` Map, `stepToTaskId` Map). New arrays (e.g., the output of `markTaskRunning` / `markTaskDone`) get a fresh build naturally. WeakMap self-invalidates when the array goes out of scope — no manual lifetime management. For the typical 5-batch/20-task run, ~80% of `buildTaskGraphIndex` calls are now cache hits.
+- **P12 (LOW)** — `mapConcurrent` now short-circuits on `length === 0` (empty array → `[]`) and `length === 1` (single item → `await fn(items[0], 0)`) without setting up the worker pool / Promise.all / counters. Common case for adaptive-plan branching and many task-graph decision points where only one ready task is dispatchable.
+- **P13 (LOW)** — Team-runner heartbeat interval bumped from 30s to 60s. The stale-reconciler threshold is 5min (`staleThresholdMs = 300_000` in crash-recovery.ts), so 60s still leaves 5 ticks of slack before a run is misidentified as stale. Halves heartbeat syscall count (1 write/30s → 1 write/60s) with no behavioral change. Worker heartbeats (1s) and the team-runner interrupt guard are unchanged.
+- **P15** — already addressed by Phase 4's P6 fix (writeProgress artifact dedup moved to O(N) Map).
+
+3 new tests lock in the changes: P14 cache-by-reference identity + cache invalidation across markTaskRunning/markTaskDone; P12 single-item + empty-item fast paths.
+
+### Performance — forward audit quick wins (Phase 4 — P6 + P9 + content-cache)
+
+Continuing the Phase 3 work. Addresses 1 MEDIUM-Low finding (P6), 1 MEDIUM finding (P9), plus a content-skip optimization for writeProgress.
+
+- **P6 (MEDIUM-LOW)** — `writeProgress()` no longer rebuilds the entire artifact-list with two redundant filter passes (one to remove the old progress entry, one with a quadratic `findIndex` dedup). Now uses a single-pass `Map<path, ArtifactDescriptor>` for O(N) replace-by-path. Also added a content-skip cache (WeakMap keyed on the manifest) — if the rendered progress content is byte-identical to the previous call (rare, but happens during back-to-back applyPolicy + executeTeamRun calls within the same millisecond), we reuse the existing artifact descriptor instead of re-running `writeArtifact` (mkdirSync + resolveRealContainedPath + redactSecrets + atomicWriteFile + readFileSync for hash).
+- **P9 (MEDIUM)** — `computeStablePrefixComponents()` gained a 60s TTL second-level cache keyed on `(cwd, step.task)` that survives `runId` boundaries. Sequential runs in the same session with the same cwd now skip the 3 awaits (`buildWorkspaceTree` + `runRetrievalCycle` + `buildKnowledgeFragment`) on cache hit. The fast path (per-run, `(cwd, step, runId)`) is unchanged so concurrent siblings in the same batch still share with zero FS access. `clearStablePrefixCache()` now also clears the cross-run layer so long-paused sessions re-warm on workspace drift promptly. 2 new tests verify the per-run short-circuit and clearStablePrefixCache behavior.
+
+### Performance — forward audit quick wins (Phase 3 — P5 + P7)
+
+Continuing the Phase 2 work. Addresses 1 MEDIUM-Severity and 1 MEDIUM-Low finding from the audit.
+
+- **P5 (MEDIUM)** — added a short-TTL (1s) cache for the target-file symlink check in `atomic-write.ts`. The previous code re-ran `lstatSync` on every `atomicWriteFile` call (`isTargetNotSymlink`), even for the common case where the file (manifest.json, tasks.json, events.jsonl inside `.crew/state/`) is a regular file repeatedly. The 1s TTL bounds the TOCTOU swap window to ~1s — same order as the existing `isSymlinkSafeDirCached` (10s). `invalidateSymlinkSafeCache(dir)` now also flushes the per-file cache entries under that dir, so the two caches stay coherent. Set `TARGET_NOT_SYMLINK_TTL_MS = 0` to disable and restore the pre-P5 always-recheck behavior. Estimated ~50% reduction in symlink-check syscalls per write burst.
+- **P7 (MEDIUM)** — migrated 5 fire-and-forget `appendEvent` call sites in `live-session-runtime.ts` to `appendEventFireAndForget`. These events (session_created, bind_extensions_error, prompt_start, prompt_error, prompt_done) had their return value ignored; the sync path acquired the event-log `sleepSync` lock, blocking SIGTERM/AbortSignal handlers during high-frequency prompt-done churn. Sites that pass `appendEvent` AS A CALLBACK to functions expecting `(eventsPath, event) => TeamEvent` (e.g. `appendTranscript`, `terminateLiveAgent`) keep the sync path — the return value is needed there. 3 new tests lock in the cache invariants: cache hit on repeated writes; explicit `invalidateSymlinkSafeCache(dir)` flushes both caches; symlink redirection is still rejected within a write.
+
+### Performance — forward audit quick wins (Phase 2 — top 4 optimizations)
+
+Follow-on to the Phase 1 quick wins; addresses the top 3 HIGH-severity findings from the deep audit (`team_20260711095156_8a8a56a3c7dff269`) plus one MEDIUM-Severity security finding from the bundled security review.
+
+- **P1 (HIGH)** — `withQueue()` in `task-graph-scheduler.ts` now returns the **same task reference** when the computed queue matches `task.graph.queue`. Eliminates the per-task per-call spread allocation that `refreshTaskGraphQueues` ran on every main-loop iteration. For a 20-task run × 5 batches, ~80% of allocations disappear (only tasks whose queue actually changed get a new object). Behavior unchanged when queues move; downstream code that relies on reference equality now gets the optimization for free.
+- **P2 (HIGH)** — `saveRunManifest` / `saveRunManifestAsync` no longer re-read `tasks.json` from disk after every manifest write. The previous code did `JSON.parse(fs.readFileSync(tasksPath, 'utf-8'))` on every call just to repopulate the cache; ~8+ manifest writes per batch iteration made this the second-hottest I/O path in team-runs. Fix captures `manifestCache.get(stateRoot)` **before** `invalidateRunCache`, reuses the cached tasks array + mtime/size on the write-through repopulation. Cache miss behavior (no prior entry) falls back to `tasks: []` — identical to the pre-fix code.
+- **P3 (HIGH)** — `persistSingleTaskUpdate` reuses the caller-supplied `fallbackTasks` on the **first attempt** instead of calling `loadRunManifestById` again. The caller already obtained the latest tasks via `loadRunManifestById` and handed them in as `fallbackTasks`; the second load was pure waste. The CAS check (`currentMtime !== baseMtime`) catches any concurrent writer — when contention is detected, the retry path **does** call `loadRunManifestById` to pull fresh state. Disk-read cost is now paid only on actual contention, not on the common single-writer happy path. For event-heavy runs (200 events × 2 sources = ~400 calls), this removes ~400 stat+read syscalls.
+- **P10 (LOW)** — `mergeTaskUpdatesPreservingTerminal` replaced its `O(N×M)` `.find()` + `.map()` nested loops with a single-pass `Map<id, task>` index. For a 20-task run × 5-batch merger with ~10 updates per result, this reduces from `O(50×20) = 1000` ops to `O(120)`. Behavior is unchanged: skipped updates (per `shouldMergeTaskUpdate`) still leave the existing task in place, and the reassembly preserves original `base` order so downstream snapshots stay stable.
+- **SEC-M1 (MEDIUM)** — `validateEndpoint` in `otlp-exporter.ts` previously checked only `::1` and `fd`/`fc` IPv6 prefixes. Added defense for `fe80::/10` (link-local), `fec0::/10` (deprecated site-local), `ff00::/8` (multicast), `::` (unspecified), and `::ffff:x.x.x.x` (IPv4-mapped — falls through to IPv4 private rules if checked). Added 5 unit tests covering each new prefix. Closes the SSRF gap flagged in the security review.
+
 ### Performance — forward audit quick wins (Phase 1)
 
 - **NEW-M1** — memoize stable prompt prefix per `(cwd, step, runId)`. The stable prefix (workspace tree + retrieval + knowledge + skills) was recomputed independently for every task in a parallel batch — identical work for siblings. `renderTaskPrompt` already separates `stablePrefix` from `dynamicSuffix`; pre-warm once per unique cwd in the team-runner batch loop. `clearStablePrefixCache()` in `finally` (matches the existing `flushEventLogBuffer` cleanup pattern) so it does not grow unbounded across runs in a long-lived session. (`08d1a64`)