pi-crew 0.9.20 → 0.9.22

package/CHANGELOG.md

@@ -1,5 +1,69 @@
 # Changelog
 
+## [v0.9.22] — fix: batch buffered event-log flush (25-85x faster) (2026-07-07)
+
+P0 follow-up to v0.9.21 event batching. The buffered flush path was calling `appendEventInsideLock` per-event, which triggered an `fsyncSync` per queued event. This defeated the entire point of batching — for 100 buffered events the flush took ~3s on tmpfs instead of the expected ~20ms.
+
+### Highlights
+
+- **Buffered flush now coalesces fsyncs.** New `appendEventBatchInsideLock()` computes metadata for the whole queue, then performs 1 `appendFileSync` + 1 `fsyncSync` + 1 `persistSequence` + 1 cache update at the end. Resolves all queued promises with their finalized events.
+- **Side fixes for batch correctness.** `nextSequence()` cannot be called per-event in Phase 1 (no writes between calls → all events get the same seq). Now computes `startingSeq` once and increments locally. `setTimeout` callback no longer returns a floating Promise from `flushOneEventLogBuffer`. `process.on('exit')` guards `flushEventLogBuffer()` with a size > 0 check to avoid creating a new floating Promise on every exit.
+
+### Performance
+
+| Workload | Before | After | Speedup |
+|---|---|---|---|
+| 100 buffered events | 3,314 ms | 132 ms | 25x |
+| 1,000 buffered events | 29,205 ms | 343 ms | 85x |
+
+### Verification
+
+| Gate | Result |
+|---|---|
+| `event-log-async.test.ts` | ✅ 16/16 pass |
+| `event-log-batch.test.ts` | ✅ all pass |
+| `event-log-buffered.test.ts` | ✅ seq uniqueness restored |
+| `event-log-race.test.ts` | ✅ all pass |
+| `event-log-leak.test.ts` H2 | ✅ pass |
+| `event-log-leak.test.ts` H1/H3 | ⚠️ cancelled under `--test-force-exit` (node:test framework detects pending work; tracked separately) |
+| `per-task-budget.test.ts`, `token-counter.test.ts` | ✅ all pass |
+
+### Files
+
+- MOD: `src/state/event-log.ts` — added `appendEventBatchInsideLock()`, fixed `nextSequence()` batching, fixed floating Promise in `setTimeout` callback, guarded `process.on('exit')` flush
+
+---
+
+## [v0.9.21] — P0 performance fixes: event batching, async worktree, per-task budget, token counter (2026-07-07)
+
+Four P0 performance fixes identified by a parallel performance review (4 subagents: runtime, token efficiency, architecture, security overhead). All fixes are backward-compatible; no breaking changes.
+
+### Highlights
+
+- **Event log batching (P0).** The event log now uses a 50ms flush interval instead of immediate per-event writes. This reduces fsync calls from 1 per event to 1 per batch. New helpers: `appendEventBuffered()`, `flushEventBuffer()`. Force flush on run completion and shutdown. `src/state/event-log.ts`; 12 tests in `test/unit/event-log-batch.test.ts`.
+
+- **Async worktree operations (P0).** `execFileSync` → async `execFile` for non-blocking I/O. `findGitRoot()` cached via Map to avoid repeated disk reads. `assertCleanLeader()` uses async execFile. Dynamic workflow context updated to use async worktree. `src/worktree/worktree-manager.ts`, `src/runtime/task-runner.ts`, `src/runtime/dynamic-workflow-context.ts`; 7 tests in `test/unit/worktree-async.test.ts`.
+
+- **Per-task token budget enforcement (P0).** `executeTeamRun()` now enforces `budgetTotal`/`budgetAbort` thresholds. Budget warning and abort events emitted when limits exceeded. `TeamRunManifest` includes `budgetTotal`, `budgetWarning`, `budgetAbort`, `budgetUnlimited` fields. Budget propagation chain: MCP tool → handleRun → executeTeamRun → manifest persistence. `src/runtime/team-runner.ts`, `src/extension/team-tool/run.ts`, `src/state/types.ts`; 14 tests in `test/unit/per-task-budget.test.ts`.
+
+- **Accurate token counting (P0).** New `token-counter.ts` with `ceil(alpha/4) + punctuation` formula. Replaces `char/4` heuristic for better accuracy on code. Single-pass O(n) algorithm, no dependencies. Tool output pruner uses `countTokens()` for accurate pruning. `src/utils/token-counter.ts`, `src/runtime/tool-output-pruner.ts`; 9 tests in `test/unit/token-counter.test.ts`.
+
+### Verification
+
+| Gate | Result |
+|---|---|
+| All unit tests (21 files) | ✅ 187/187 pass |
+| New P0 tests (4 files) | ✅ 67/67 pass |
+| Live E2E (fast-fix workflow) | ✅ 3/3 tasks, budget propagation verified |
+| Extension load | ✅ |
+
+### Files
+
+- NEW: `src/utils/token-counter.ts`, `test/unit/event-log-batch.test.ts`, `test/unit/per-task-budget.test.ts`, `test/unit/token-counter.test.ts`, `test/unit/worktree-async.test.ts`
+- MOD: `src/state/event-log.ts`, `src/worktree/worktree-manager.ts`, `src/runtime/team-runner.ts`, `src/runtime/task-runner.ts`, `src/runtime/dynamic-workflow-context.ts`, `src/runtime/tool-output-pruner.ts`, `src/extension/team-tool/run.ts`, `src/extension/cross-extension-rpc.ts`, `src/runtime/background-runner.ts`, `src/state/types.ts`
+
+---
+
 ## [v0.9.20] — security hardening: RPC HMAC auth + per-task API key scoping + safe-bash whitelist (2026-07-06)
 
 Three defense-in-depth security upgrades distilled from cross-source research (52+ repos) and the H-1/H-2/H-6 audit findings. All three are additive (no breaking changes); two are opt-in via env vars, one is default-on.

package/dist/build-meta.json

@@ -6351,7 +6351,7 @@
       "format": "esm"
     },
     "src/state/event-log.ts": {
-      "bytes": 37533,
+      "bytes": 44845,
       "imports": [
         {
           "path": "node:crypto",
@@ -7233,7 +7233,7 @@
       "format": "esm"
     },
     "src/utils/env-filter.ts": {
-      "bytes": 4485,
+      "bytes": 6523,
       "imports": [
         {
           "path": "src/utils/redaction.ts",
@@ -7341,7 +7341,7 @@
       "format": "esm"
     },
     "src/runtime/child-pi.ts": {
-      "bytes": 61943,
+      "bytes": 60312,
       "imports": [
         {
           "path": "node:child_process",
@@ -14125,11 +14125,22 @@
       ],
       "format": "esm"
     },
-    "src/runtime/tool-output-pruner.ts": {
-      "bytes": 12289,
+    "src/utils/token-counter.ts": {
+      "bytes": 2162,
       "imports": [],
       "format": "esm"
     },
+    "src/runtime/tool-output-pruner.ts": {
+      "bytes": 12320,
+      "imports": [
+        {
+          "path": "src/utils/token-counter.ts",
+          "kind": "import-statement",
+          "original": "../utils/token-counter.ts"
+        }
+      ],
+      "format": "esm"
+    },
     "src/runtime/task-output-context.ts": {
       "bytes": 26832,
       "imports": [
@@ -14358,7 +14369,7 @@
       "format": "esm"
     },
     "src/worktree/worktree-manager.ts": {
-      "bytes": 27468,
+      "bytes": 40499,
       "imports": [
         {
           "path": "node:child_process",
@@ -14380,6 +14391,11 @@
           "kind": "import-statement",
           "external": true
         },
+        {
+          "path": "node:util",
+          "kind": "import-statement",
+          "external": true
+        },
         {
           "path": "src/config/config.ts",
           "kind": "import-statement",
@@ -14794,7 +14810,7 @@
       "format": "esm"
     },
     "src/runtime/task-runner.ts": {
-      "bytes": 54685,
+      "bytes": 54735,
       "imports": [
         {
           "path": "node:fs",
@@ -15103,7 +15119,7 @@
       "format": "esm"
     },
     "src/runtime/team-runner.ts": {
-      "bytes": 70267,
+      "bytes": 75896,
       "imports": [
         {
           "path": "node:fs",
@@ -15631,7 +15647,7 @@
       "format": "esm"
     },
     "src/runtime/dynamic-workflow-context.ts": {
-      "bytes": 40912,
+      "bytes": 40944,
       "imports": [
         {
           "path": "node:crypto",
@@ -15768,7 +15784,7 @@
       "format": "esm"
     },
     "src/extension/team-tool/run.ts": {
-      "bytes": 42914,
+      "bytes": 44253,
       "imports": [
         {
           "path": "src/agents/discover-agents.ts",
@@ -17096,14 +17112,30 @@
       ],
       "format": "esm"
     },
+    "src/extension/rpc-hmac.ts": {
+      "bytes": 7500,
+      "imports": [
+        {
+          "path": "node:crypto",
+          "kind": "import-statement",
+          "external": true
+        }
+      ],
+      "format": "esm"
+    },
     "src/extension/cross-extension-rpc.ts": {
-      "bytes": 10489,
+      "bytes": 10981,
       "imports": [
         {
           "path": "src/utils/safe-paths.ts",
           "kind": "import-statement",
           "original": "../utils/safe-paths.ts"
         },
+        {
+          "path": "src/extension/rpc-hmac.ts",
+          "kind": "import-statement",
+          "original": "./rpc-hmac.ts"
+        },
         {
           "path": "src/runtime/live-control-realtime.ts",
           "kind": "import-statement",
@@ -18166,7 +18198,7 @@
       "imports": [],
       "exports": [],
       "inputs": {},
-      "bytes": 6427881
+      "bytes": 6479322
     },
     "dist/index.mjs": {
       "imports": [
@@ -19205,6 +19237,11 @@
           "kind": "import-statement",
           "external": true
         },
+        {
+          "path": "node:util",
+          "kind": "import-statement",
+          "external": true
+        },
         {
           "path": "node:fs",
           "kind": "import-statement",
@@ -19545,6 +19582,11 @@
           "kind": "import-statement",
           "external": true
         },
+        {
+          "path": "node:crypto",
+          "kind": "import-statement",
+          "external": true
+        },
         {
           "path": "@earendil-works/pi-tui",
           "kind": "import-statement",
@@ -20349,7 +20391,7 @@
           "bytesInOutput": 4478
         },
         "src/state/event-log.ts": {
-          "bytesInOutput": 22276
+          "bytesInOutput": 26234
         },
         "src/hooks/registry.ts": {
           "bytesInOutput": 4422
@@ -20448,7 +20490,7 @@
           "bytesInOutput": 291
         },
         "src/utils/env-filter.ts": {
-          "bytesInOutput": 2272
+          "bytesInOutput": 3511
         },
         "src/runtime/compact-pipeline.ts": {
           "bytesInOutput": 517
@@ -20487,7 +20529,7 @@
           "bytesInOutput": 1642
         },
         "src/runtime/child-pi.ts": {
-          "bytesInOutput": 43955
+          "bytesInOutput": 41851
         },
         "src/runtime/heartbeat-gradient.ts": {
           "bytesInOutput": 953
@@ -21377,8 +21419,11 @@
         "src/runtime/goal-achievement.ts": {
           "bytesInOutput": 2922
         },
+        "src/utils/token-counter.ts": {
+          "bytesInOutput": 685
+        },
         "src/runtime/tool-output-pruner.ts": {
-          "bytesInOutput": 5528
+          "bytesInOutput": 5527
         },
         "src/runtime/task-output-context.ts": {
           "bytesInOutput": 12672
@@ -21423,7 +21468,7 @@
           "bytesInOutput": 2538
         },
         "src/worktree/worktree-manager.ts": {
-          "bytesInOutput": 20059
+          "bytesInOutput": 21100
         },
         "src/runtime/event-stream-bridge.ts": {
           "bytesInOutput": 2321
@@ -21489,7 +21534,7 @@
           "bytesInOutput": 6123
         },
         "src/runtime/task-runner.ts": {
-          "bytesInOutput": 42442
+          "bytesInOutput": 42475
         },
         "src/runtime/workflow-state.ts": {
           "bytesInOutput": 2471
@@ -21507,7 +21552,7 @@
           "bytesInOutput": 3398
         },
         "src/runtime/team-runner.ts": {
-          "bytesInOutput": 53388
+          "bytesInOutput": 57404
         },
         "src/runtime/pipeline-runner.ts": {
           "bytesInOutput": 11755
@@ -21558,13 +21603,13 @@
           "bytesInOutput": 1460
         },
         "src/runtime/dynamic-workflow-context.ts": {
-          "bytesInOutput": 20185
+          "bytesInOutput": 20207
         },
         "src/runtime/dynamic-workflow-runner.ts": {
           "bytesInOutput": 6259
         },
         "src/extension/team-tool/run.ts": {
-          "bytesInOutput": 33569
+          "bytesInOutput": 34875
         },
         "src/extension/team-tool.ts": {
           "bytesInOutput": 35293
@@ -21672,7 +21717,7 @@
           "bytesInOutput": 6698
         },
         "src/observability/exporters/otlp-exporter.ts": {
-          "bytesInOutput": 7427
+          "bytesInOutput": 7442
         },
         "src/observability/metrics-primitives.ts": {
           "bytesInOutput": 5869
@@ -21753,7 +21798,10 @@
           "bytesInOutput": 1204
         },
         "src/extension/cross-extension-rpc.ts": {
-          "bytesInOutput": 7789
+          "bytesInOutput": 8532
+        },
+        "src/extension/rpc-hmac.ts": {
+          "bytesInOutput": 3514
         },
         "src/extension/message-renderers.ts": {
           "bytesInOutput": 2457
@@ -21783,7 +21831,7 @@
           "bytesInOutput": 81
         }
       },
-      "bytes": 3083229
+      "bytes": 3097798
     }
   }
 }