pi-crew 0.9.51 → 0.9.53

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (611) hide show
  1. package/CHANGELOG.md +53 -0
  2. package/README.md +216 -20
  3. package/dist/build-meta.json +2090 -1638
  4. package/dist/index.mjs +18057 -17119
  5. package/docs/actions-reference.md +431 -1
  6. package/index.ts +39 -10
  7. package/package.json +19 -9
  8. package/schema.json +41 -0
  9. package/scripts/check-bundle-size.mjs +38 -0
  10. package/scripts/test-runner.mjs +11 -1
  11. package/dist/index.mjs.map +0 -7
  12. package/docs/FEATURE_INTAKE.md +0 -126
  13. package/docs/HARNESS.md +0 -87
  14. package/docs/HARNESS_BACKLOG.md +0 -89
  15. package/docs/PROJECT_REVIEW.md +0 -270
  16. package/docs/PROJECT_REVIEW_FIXES.md +0 -342
  17. package/docs/PROJECT_REVIEW_ROUND4.md +0 -156
  18. package/docs/PROJECT_REVIEW_ROUND5.md +0 -86
  19. package/docs/REVIEW-FINDINGS-2026-06 +0 -357
  20. package/docs/REVIEW-FINDINGS-2026-07-CORE.md +0 -139
  21. package/docs/REVIEW-FINDINGS-2026-07.md +0 -125
  22. package/docs/TEST_MATRIX.md +0 -51
  23. package/docs/archive/README.md +0 -91
  24. package/docs/archive/followup-review-round3-2026-05-12.md +0 -342
  25. package/docs/archive/followup-review-round4-2026-05-13.md +0 -107
  26. package/docs/archive/pi-crew-bugs.md +0 -961
  27. package/docs/archive/pi-crew-test-final.md +0 -120
  28. package/docs/archive/pi-crew-test-results.md +0 -260
  29. package/docs/archive/pi-crew-test-round2.md +0 -136
  30. package/docs/archive/pi-crew-test-round4.md +0 -100
  31. package/docs/archive/pi-crew-test-round5.md +0 -70
  32. package/docs/archive/pi-crew-test-round6.md +0 -110
  33. package/docs/archive/pi-crew-v0.5.10-audit-fix-plan.md +0 -60
  34. package/docs/archive/pi-crew-v0.5.11-audit-fix-plan.md +0 -92
  35. package/docs/archive/pi-crew-v0.5.12-audit-fix-plan.md +0 -76
  36. package/docs/archive/pi-crew-v0.5.13-audit-fix-plan.md +0 -75
  37. package/docs/archive/pi-crew-v0.5.14-audit-fix-plan.md +0 -75
  38. package/docs/archive/pi-crew-v0.5.16-audit-fix-plan.md +0 -35
  39. package/docs/archive/pi-crew-v0.5.17-audit-fix-plan.md +0 -80
  40. package/docs/archive/pi-crew-v0.5.5-audit-fix-plan.md +0 -133
  41. package/docs/archive/pi-crew-v0.5.9-audit-fix-plan.md +0 -88
  42. package/docs/archive/pi-mono-opportunities.md +0 -969
  43. package/docs/archive/pi-mono-review.md +0 -291
  44. package/docs/archive/pi-subagent4-comparison.md +0 -261
  45. package/docs/archive/pi-subagents3-deep-analysis.md +0 -508
  46. package/docs/bugs/bug-021-notification-badge-counter-misleading.md +0 -293
  47. package/docs/bugs/bug-023-chain-windows-path-resolution.md +0 -106
  48. package/docs/bugs/bug-quota-display-truncation.md +0 -223
  49. package/docs/bugs/cross-session-notification-leakage.md +0 -82
  50. package/docs/code-review-2026-05-11.md +0 -592
  51. package/docs/coding-agent-optimization.md +0 -268
  52. package/docs/comparison-pi-subagents-vs-pi-crew.md +0 -303
  53. package/docs/decisions/0001-durable-state.md +0 -41
  54. package/docs/decisions/0002-child-process-for-async.md +0 -42
  55. package/docs/decisions/0003-depth-guard.md +0 -36
  56. package/docs/decisions/0004-execfile-over-exec.md +0 -34
  57. package/docs/decisions/0005-no-parameter-properties.md +0 -49
  58. package/docs/decisions/0006-publish-bundled-esm.md +0 -63
  59. package/docs/decisions/0007-active-run-binary-index.md +0 -54
  60. package/docs/decisions/0008-child-pi-warm-pool.md +0 -61
  61. package/docs/decisions/2026-07-21-broker-phase4-default-on.md +0 -77
  62. package/docs/decisions/2026-07-21-broker-windows-perms.md +0 -91
  63. package/docs/decisions/2026-07-22-broker-phase4-gated-on.md +0 -99
  64. package/docs/decisions/2026-07-24-oidc-trusted-publishing.md +0 -112
  65. package/docs/decisions/2026-07-26-c6-mascot-visibility-not-wired.md +0 -84
  66. package/docs/decisions/README.md +0 -26
  67. package/docs/deep-review-report.md +0 -384
  68. package/docs/distillation/cybersecurity-patterns.md +0 -294
  69. package/docs/feature-analysis-subagent4.md +0 -305
  70. package/docs/fix-plan-disabletools-exit-null.md +0 -219
  71. package/docs/fixes/BATCH_A_H1_H2.md +0 -86
  72. package/docs/fixes/bug-006-foreground-cancel-concurrent.md +0 -78
  73. package/docs/fixes/bug-007-async-notifier-stale-ctx.md +0 -112
  74. package/docs/fixes/bug-008-child-process-silent-timeout.md +0 -100
  75. package/docs/fixes/bug-009-executor-yield-limit-needs-attention.md +0 -75
  76. package/docs/fixes/bug-010-child-process-api-key-filtered.md +0 -109
  77. package/docs/fixes/bug-011-spawn-pi-enoent.md +0 -92
  78. package/docs/fixes/bug-012-essential-env-stripped.md +0 -89
  79. package/docs/fixes/bug-013-background-runner-death.md +0 -84
  80. package/docs/fixes/bug-014-infinite-retry-loop-needs-attention.md +0 -82
  81. package/docs/fixes/bug-015-background-runner-sigterm.md +0 -65
  82. package/docs/fixes/bug-017-background-runner-session-shutdown.md +0 -66
  83. package/docs/fixes/bug-017-background-runner-sigkill-double-fork.md +0 -28
  84. package/docs/fixes/bug-018-child-pi-worker-stdin-hang.md +0 -61
  85. package/docs/fixes/bug-019-phantom-runs-temp-workspace.md +0 -52
  86. package/docs/fixes/bug-020-infinite-retry-loop-needs-attention.md +0 -47
  87. package/docs/fixes/v0.9.10/locks-fix-verify.md +0 -3
  88. package/docs/fixes/v0.9.10/smoke-test.md +0 -12
  89. package/docs/followup-plan-2026-05-12.md +0 -463
  90. package/docs/followup-review-2026-05-12.md +0 -297
  91. package/docs/goals.md +0 -59
  92. package/docs/implementation-plan-top3.md +0 -333
  93. package/docs/issue-29-analysis.md +0 -189
  94. package/docs/live-mailbox-runtime.md +0 -36
  95. package/docs/migration/atomic-write-v2-migration.md +0 -299
  96. package/docs/next-upgrade-roadmap.md +0 -808
  97. package/docs/oh-my-pi-research.md +0 -509
  98. package/docs/optimization-plan-2026-07.md +0 -366
  99. package/docs/optimization-plan.md +0 -644
  100. package/docs/patterns/command-agent-skill.md +0 -71
  101. package/docs/perf/baseline-2026-05.md +0 -113
  102. package/docs/perf/final-report-2026-05.md +0 -206
  103. package/docs/perf/optimization-plan-2026-07-verified.md +0 -396
  104. package/docs/perf/performance-audit-report-2026-07.md +0 -948
  105. package/docs/perf/performance-review-2026-07.md +0 -287
  106. package/docs/perf/sprint-1-report.md +0 -71
  107. package/docs/perf/sprint-2-report.md +0 -81
  108. package/docs/perf/sprint-2.5-report.md +0 -53
  109. package/docs/perf/sprint-3-report.md +0 -36
  110. package/docs/perf/sprint-4-report.md +0 -47
  111. package/docs/perf/sprint-5-report.md +0 -51
  112. package/docs/perf/sprint-6-report.md +0 -94
  113. package/docs/perf/sprint-7-report.md +0 -74
  114. package/docs/perf/upgrade-plan-2026-05.md +0 -147
  115. package/docs/performance-optimization-execution-plan.md +0 -520
  116. package/docs/phase4-triage.md +0 -58
  117. package/docs/pi-crew-investigation-report.md +0 -411
  118. package/docs/product/README.md +0 -31
  119. package/docs/product/platform.md +0 -27
  120. package/docs/product/runtime-safety.md +0 -37
  121. package/docs/product/team-run.md +0 -39
  122. package/docs/product/team-tool.md +0 -37
  123. package/docs/runtime-analysis-child-vs-live.md +0 -171
  124. package/docs/runtime-migration-in-process-analysis.md +0 -250
  125. package/docs/skills/REFERENCE.md +0 -138
  126. package/docs/stories/README.md +0 -32
  127. package/docs/stories/US-DEPS-major-upgrade.md +0 -62
  128. package/docs/stories/backlog.md +0 -36
  129. package/docs/superpowers/plans/2026-06-09-fallow-patterns-adoption.md +0 -1270
  130. package/docs/templates/decision.md +0 -27
  131. package/docs/templates/story.md +0 -44
  132. package/docs/templates/validation-report.md +0 -32
  133. package/docs/ui-optimization-plan.md +0 -447
  134. package/src/adapters/claude-adapter.ts +0 -23
  135. package/src/adapters/codex-adapter.ts +0 -21
  136. package/src/adapters/cursor-adapter.ts +0 -17
  137. package/src/adapters/export-util.ts +0 -143
  138. package/src/adapters/index.ts +0 -15
  139. package/src/adapters/registry.ts +0 -18
  140. package/src/adapters/types.ts +0 -23
  141. package/src/agents/agent-config.ts +0 -194
  142. package/src/agents/agent-search.ts +0 -98
  143. package/src/agents/agent-serializer.ts +0 -38
  144. package/src/agents/discover-agents.ts +0 -623
  145. package/src/benchmark/benchmark-runner.ts +0 -313
  146. package/src/benchmark/feedback-loop.ts +0 -73
  147. package/src/config/config.ts +0 -1269
  148. package/src/config/defaults.ts +0 -193
  149. package/src/config/drift-detector.ts +0 -290
  150. package/src/config/markers.ts +0 -330
  151. package/src/config/resilient-parser.ts +0 -117
  152. package/src/config/role-tools.ts +0 -118
  153. package/src/config/suggestions.ts +0 -75
  154. package/src/config/types.ts +0 -278
  155. package/src/errors.ts +0 -194
  156. package/src/extension/action-suggestions.ts +0 -117
  157. package/src/extension/async-notifier.ts +0 -179
  158. package/src/extension/autonomous-policy.ts +0 -209
  159. package/src/extension/command-completions.ts +0 -120
  160. package/src/extension/context-status-injection.ts +0 -185
  161. package/src/extension/crew-autocomplete.ts +0 -131
  162. package/src/extension/crew-cleanup.ts +0 -168
  163. package/src/extension/crew-input-router.ts +0 -93
  164. package/src/extension/crew-shortcuts.ts +0 -87
  165. package/src/extension/crew-vibes/cat-frames.ts +0 -18
  166. package/src/extension/crew-vibes/config.ts +0 -195
  167. package/src/extension/crew-vibes/figures.ts +0 -101
  168. package/src/extension/crew-vibes/font-detect.ts +0 -71
  169. package/src/extension/crew-vibes/footer.ts +0 -292
  170. package/src/extension/crew-vibes/index.ts +0 -432
  171. package/src/extension/crew-vibes/provider-usage.ts +0 -396
  172. package/src/extension/crew-vibes/render.ts +0 -206
  173. package/src/extension/crew-vibes/speed.ts +0 -286
  174. package/src/extension/cross-extension-rpc.ts +0 -350
  175. package/src/extension/help.ts +0 -62
  176. package/src/extension/import-index.ts +0 -74
  177. package/src/extension/knowledge-injection.ts +0 -402
  178. package/src/extension/management.ts +0 -571
  179. package/src/extension/message-renderers.ts +0 -108
  180. package/src/extension/notification-router.ts +0 -152
  181. package/src/extension/notification-sink.ts +0 -54
  182. package/src/extension/pi-api.ts +0 -56
  183. package/src/extension/plan-orchestrate.ts +0 -302
  184. package/src/extension/project-init.ts +0 -170
  185. package/src/extension/register.ts +0 -113
  186. package/src/extension/registration/artifact-cleanup.ts +0 -20
  187. package/src/extension/registration/command-registration.ts +0 -58
  188. package/src/extension/registration/command-utils.ts +0 -58
  189. package/src/extension/registration/commands.ts +0 -1228
  190. package/src/extension/registration/compaction-guard.ts +0 -333
  191. package/src/extension/registration/context-builder.ts +0 -135
  192. package/src/extension/registration/crash-recovery-cache.ts +0 -54
  193. package/src/extension/registration/foreground-run-controller.ts +0 -238
  194. package/src/extension/registration/hook-registration.ts +0 -119
  195. package/src/extension/registration/lazy-configurers.ts +0 -124
  196. package/src/extension/registration/lifecycle-handlers.ts +0 -923
  197. package/src/extension/registration/lifecycle.ts +0 -259
  198. package/src/extension/registration/observability.ts +0 -322
  199. package/src/extension/registration/registration-types.ts +0 -158
  200. package/src/extension/registration/runtime-cleanup.ts +0 -230
  201. package/src/extension/registration/subagent-helpers.ts +0 -125
  202. package/src/extension/registration/subagent-manager-setup.ts +0 -330
  203. package/src/extension/registration/subagent-tools.ts +0 -456
  204. package/src/extension/registration/team-tool.ts +0 -220
  205. package/src/extension/registration/tool-registration.ts +0 -50
  206. package/src/extension/registration/ui.ts +0 -172
  207. package/src/extension/registration/viewers.ts +0 -121
  208. package/src/extension/registration/wire-cross-extension.ts +0 -37
  209. package/src/extension/result-watcher.ts +0 -139
  210. package/src/extension/rpc-hmac.ts +0 -256
  211. package/src/extension/run-bundle-schema.ts +0 -104
  212. package/src/extension/run-export.ts +0 -97
  213. package/src/extension/run-import.ts +0 -168
  214. package/src/extension/run-index.ts +0 -131
  215. package/src/extension/run-maintenance.ts +0 -190
  216. package/src/extension/session-summary.ts +0 -18
  217. package/src/extension/team-manager-command.ts +0 -153
  218. package/src/extension/team-onboard.ts +0 -177
  219. package/src/extension/team-recommendation.ts +0 -288
  220. package/src/extension/team-tool/anchor.ts +0 -172
  221. package/src/extension/team-tool/api.ts +0 -1244
  222. package/src/extension/team-tool/auto-summarize.ts +0 -142
  223. package/src/extension/team-tool/cache-control.ts +0 -19
  224. package/src/extension/team-tool/cancel.ts +0 -337
  225. package/src/extension/team-tool/chain-dispatch.ts +0 -95
  226. package/src/extension/team-tool/chain-executor.ts +0 -379
  227. package/src/extension/team-tool/config-patch.ts +0 -50
  228. package/src/extension/team-tool/context.ts +0 -149
  229. package/src/extension/team-tool/destructive-gate.ts +0 -52
  230. package/src/extension/team-tool/doctor.ts +0 -431
  231. package/src/extension/team-tool/explain.ts +0 -282
  232. package/src/extension/team-tool/failure-patterns.ts +0 -118
  233. package/src/extension/team-tool/goal-wrap.ts +0 -330
  234. package/src/extension/team-tool/goal.ts +0 -556
  235. package/src/extension/team-tool/handle-schedule.ts +0 -332
  236. package/src/extension/team-tool/handle-settings.ts +0 -520
  237. package/src/extension/team-tool/health-monitor.ts +0 -481
  238. package/src/extension/team-tool/inspect.ts +0 -88
  239. package/src/extension/team-tool/intent-policy.ts +0 -45
  240. package/src/extension/team-tool/lifecycle-actions.ts +0 -615
  241. package/src/extension/team-tool/orchestrate.ts +0 -98
  242. package/src/extension/team-tool/parallel-dispatch.ts +0 -183
  243. package/src/extension/team-tool/plan.ts +0 -50
  244. package/src/extension/team-tool/respond.ts +0 -150
  245. package/src/extension/team-tool/run-not-found.ts +0 -49
  246. package/src/extension/team-tool/run.ts +0 -1035
  247. package/src/extension/team-tool/status.ts +0 -260
  248. package/src/extension/team-tool/workflow-manage.ts +0 -261
  249. package/src/extension/team-tool-types.ts +0 -26
  250. package/src/extension/team-tool.ts +0 -1116
  251. package/src/extension/tool-result.ts +0 -16
  252. package/src/extension/validate-resources.ts +0 -108
  253. package/src/hooks/registry.ts +0 -200
  254. package/src/hooks/types.ts +0 -69
  255. package/src/i18n.ts +0 -212
  256. package/src/observability/correlation.ts +0 -50
  257. package/src/observability/event-bus.ts +0 -86
  258. package/src/observability/event-to-metric.ts +0 -170
  259. package/src/observability/exporters/adapter.ts +0 -27
  260. package/src/observability/exporters/otlp-exporter.ts +0 -356
  261. package/src/observability/exporters/prometheus-exporter.ts +0 -54
  262. package/src/observability/metric-registry.ts +0 -100
  263. package/src/observability/metric-retention.ts +0 -64
  264. package/src/observability/metric-sink.ts +0 -99
  265. package/src/observability/metrics-primitives.ts +0 -219
  266. package/src/plugins/plugin-define.ts +0 -6
  267. package/src/plugins/plugin-registry.ts +0 -32
  268. package/src/plugins/plugins/index.ts +0 -3
  269. package/src/plugins/plugins/nextjs.ts +0 -19
  270. package/src/plugins/plugins/vite.ts +0 -10
  271. package/src/plugins/plugins/vitest.ts +0 -9
  272. package/src/prompt/prompt-runtime.ts +0 -380
  273. package/src/runtime/adaptive-plan.ts +0 -563
  274. package/src/runtime/agent-control.ts +0 -215
  275. package/src/runtime/agent-memory.ts +0 -81
  276. package/src/runtime/agent-observability.ts +0 -122
  277. package/src/runtime/anchor-manager.ts +0 -475
  278. package/src/runtime/async-marker.ts +0 -32
  279. package/src/runtime/async-runner.ts +0 -381
  280. package/src/runtime/attention-events.ts +0 -28
  281. package/src/runtime/auto-summarize.ts +0 -344
  282. package/src/runtime/background-runner.ts +0 -837
  283. package/src/runtime/batch-barrier.ts +0 -147
  284. package/src/runtime/broker-issuer.ts +0 -37
  285. package/src/runtime/cancellation-token.ts +0 -99
  286. package/src/runtime/cancellation.ts +0 -99
  287. package/src/runtime/capability-inventory.ts +0 -137
  288. package/src/runtime/chain-parser.ts +0 -237
  289. package/src/runtime/chain-runner.ts +0 -606
  290. package/src/runtime/checkpoint.ts +0 -291
  291. package/src/runtime/child-pi-constants.ts +0 -42
  292. package/src/runtime/child-pi-kill.ts +0 -180
  293. package/src/runtime/child-pi-pool.ts +0 -68
  294. package/src/runtime/child-pi-spawn.ts +0 -287
  295. package/src/runtime/child-pi-steering.ts +0 -128
  296. package/src/runtime/child-pi-streams.ts +0 -296
  297. package/src/runtime/child-pi-transcript.ts +0 -169
  298. package/src/runtime/child-pi.ts +0 -1105
  299. package/src/runtime/coalesce-tasks.ts +0 -268
  300. package/src/runtime/code-summary.ts +0 -292
  301. package/src/runtime/command-trace.ts +0 -105
  302. package/src/runtime/compact-pipeline.ts +0 -56
  303. package/src/runtime/compact-stages/ansi-strip-stage.ts +0 -25
  304. package/src/runtime/compact-stages/blank-collapse-stage.ts +0 -31
  305. package/src/runtime/compact-stages/deduplicate-stage.ts +0 -34
  306. package/src/runtime/compact-stages/head-snap-stage.ts +0 -57
  307. package/src/runtime/compact-stages/index.ts +0 -23
  308. package/src/runtime/compact-stages/tail-capture-stage.ts +0 -77
  309. package/src/runtime/compact-stages/truncation-stage.ts +0 -74
  310. package/src/runtime/compaction-summary.ts +0 -278
  311. package/src/runtime/completion-guard.ts +0 -207
  312. package/src/runtime/concurrency.ts +0 -58
  313. package/src/runtime/crash-classification.ts +0 -229
  314. package/src/runtime/crash-recovery.ts +0 -577
  315. package/src/runtime/crew-agent-records.ts +0 -593
  316. package/src/runtime/crew-agent-runtime.ts +0 -62
  317. package/src/runtime/crew-broker-child.ts +0 -88
  318. package/src/runtime/crew-broker-client.ts +0 -673
  319. package/src/runtime/crew-broker-tokens.ts +0 -84
  320. package/src/runtime/crew-broker.ts +0 -1276
  321. package/src/runtime/crew-hooks.ts +0 -227
  322. package/src/runtime/cross-extension-rpc.ts +0 -143
  323. package/src/runtime/custom-tools/irc-tool.ts +0 -282
  324. package/src/runtime/custom-tools/submit-result-tool.ts +0 -102
  325. package/src/runtime/deadletter.ts +0 -47
  326. package/src/runtime/delivery-coordinator.ts +0 -211
  327. package/src/runtime/delta-conflict.ts +0 -348
  328. package/src/runtime/deterministic-ast.ts +0 -161
  329. package/src/runtime/diagnostic-export.ts +0 -171
  330. package/src/runtime/direct-run.ts +0 -45
  331. package/src/runtime/dwf-state-store.ts +0 -102
  332. package/src/runtime/dynamic-workflow-context.ts +0 -1016
  333. package/src/runtime/dynamic-workflow-runner.ts +0 -310
  334. package/src/runtime/effectiveness.ts +0 -90
  335. package/src/runtime/errors/crew-errors.ts +0 -162
  336. package/src/runtime/event-stream-bridge.ts +0 -98
  337. package/src/runtime/foreground-control.ts +0 -193
  338. package/src/runtime/foreground-watchdog.ts +0 -132
  339. package/src/runtime/global-worker-cap.ts +0 -96
  340. package/src/runtime/goal-achievement.ts +0 -148
  341. package/src/runtime/goal-evaluator.ts +0 -361
  342. package/src/runtime/goal-loop-runner.ts +0 -828
  343. package/src/runtime/goal-state-store.ts +0 -219
  344. package/src/runtime/green-contract.ts +0 -55
  345. package/src/runtime/group-join.ts +0 -267
  346. package/src/runtime/handoff-manager.ts +0 -598
  347. package/src/runtime/heartbeat-gradient.ts +0 -36
  348. package/src/runtime/heartbeat-watcher.ts +0 -239
  349. package/src/runtime/hidden-handoff.ts +0 -407
  350. package/src/runtime/important-line-classifier.ts +0 -140
  351. package/src/runtime/intercom-bridge.ts +0 -187
  352. package/src/runtime/iteration-hooks.ts +0 -305
  353. package/src/runtime/live-agent-control.ts +0 -136
  354. package/src/runtime/live-agent-manager.ts +0 -704
  355. package/src/runtime/live-control-realtime.ts +0 -56
  356. package/src/runtime/live-extension-bridge.ts +0 -148
  357. package/src/runtime/live-irc.ts +0 -97
  358. package/src/runtime/live-session-health.ts +0 -108
  359. package/src/runtime/live-session-runtime.ts +0 -1139
  360. package/src/runtime/loop-gates.ts +0 -128
  361. package/src/runtime/manifest-cache.ts +0 -375
  362. package/src/runtime/mcp-proxy.ts +0 -105
  363. package/src/runtime/metric-parser.ts +0 -36
  364. package/src/runtime/model-fallback.ts +0 -409
  365. package/src/runtime/model-resolver.ts +0 -126
  366. package/src/runtime/model-scope.ts +0 -158
  367. package/src/runtime/orphan-worker-registry.ts +0 -465
  368. package/src/runtime/output-validator.ts +0 -202
  369. package/src/runtime/overflow-recovery.ts +0 -206
  370. package/src/runtime/parallel-research.ts +0 -68
  371. package/src/runtime/parallel-utils.ts +0 -160
  372. package/src/runtime/parent-guard.ts +0 -134
  373. package/src/runtime/path-overlap.ts +0 -150
  374. package/src/runtime/peer-dep.ts +0 -292
  375. package/src/runtime/per-write-validator.ts +0 -181
  376. package/src/runtime/phase-progress.ts +0 -217
  377. package/src/runtime/phase-tracker.ts +0 -385
  378. package/src/runtime/pi-args.ts +0 -655
  379. package/src/runtime/pi-json-output.ts +0 -170
  380. package/src/runtime/pi-spawn.ts +0 -273
  381. package/src/runtime/pipeline-runner.ts +0 -523
  382. package/src/runtime/plan-templates.ts +0 -201
  383. package/src/runtime/policy-engine.ts +0 -115
  384. package/src/runtime/post-checks.ts +0 -142
  385. package/src/runtime/post-exit-stdio-guard.ts +0 -104
  386. package/src/runtime/process-lifecycle.ts +0 -491
  387. package/src/runtime/process-status.ts +0 -154
  388. package/src/runtime/progress-event-coalescer.ts +0 -44
  389. package/src/runtime/progress-tracker.ts +0 -124
  390. package/src/runtime/prose-compressor.ts +0 -162
  391. package/src/runtime/recovery-recipes.ts +0 -203
  392. package/src/runtime/replace.ts +0 -570
  393. package/src/runtime/resilient-edit.ts +0 -153
  394. package/src/runtime/result-extractor.ts +0 -213
  395. package/src/runtime/retry-executor.ts +0 -100
  396. package/src/runtime/retry-runner.ts +0 -336
  397. package/src/runtime/role-permission.ts +0 -59
  398. package/src/runtime/run-coalesced-task-group.ts +0 -308
  399. package/src/runtime/run-drift.ts +0 -219
  400. package/src/runtime/run-tracker.ts +0 -116
  401. package/src/runtime/runtime-policy.ts +0 -29
  402. package/src/runtime/runtime-resolver.ts +0 -172
  403. package/src/runtime/runtime-warmup.ts +0 -160
  404. package/src/runtime/scheduler.ts +0 -353
  405. package/src/runtime/semaphore.ts +0 -141
  406. package/src/runtime/sensitive-paths.ts +0 -93
  407. package/src/runtime/session-resources.ts +0 -25
  408. package/src/runtime/session-snapshot.ts +0 -59
  409. package/src/runtime/session-usage.ts +0 -79
  410. package/src/runtime/settings-store.ts +0 -155
  411. package/src/runtime/sidechain-output.ts +0 -35
  412. package/src/runtime/single-agent-compose.ts +0 -84
  413. package/src/runtime/skill-effectiveness.ts +0 -524
  414. package/src/runtime/skill-instructions.ts +0 -391
  415. package/src/runtime/stale-reconciler.ts +0 -680
  416. package/src/runtime/stream-preview.ts +0 -184
  417. package/src/runtime/streaming-output.ts +0 -50
  418. package/src/runtime/subagent-manager.ts +0 -561
  419. package/src/runtime/subprocess-tool-registry.ts +0 -70
  420. package/src/runtime/supervisor-contact.ts +0 -64
  421. package/src/runtime/task-display.ts +0 -52
  422. package/src/runtime/task-graph-scheduler.ts +0 -227
  423. package/src/runtime/task-graph.ts +0 -290
  424. package/src/runtime/task-health.ts +0 -83
  425. package/src/runtime/task-id.ts +0 -161
  426. package/src/runtime/task-output-context.ts +0 -602
  427. package/src/runtime/task-packet.ts +0 -268
  428. package/src/runtime/task-quality.ts +0 -199
  429. package/src/runtime/task-runner/capabilities.ts +0 -78
  430. package/src/runtime/task-runner/context-retrieval.ts +0 -163
  431. package/src/runtime/task-runner/live-executor.ts +0 -227
  432. package/src/runtime/task-runner/output-splitter.ts +0 -152
  433. package/src/runtime/task-runner/progress.ts +0 -157
  434. package/src/runtime/task-runner/prompt-builder.ts +0 -278
  435. package/src/runtime/task-runner/prompt-pipeline.ts +0 -88
  436. package/src/runtime/task-runner/result-utils.ts +0 -16
  437. package/src/runtime/task-runner/retrieval-orchestrator.ts +0 -310
  438. package/src/runtime/task-runner/run-projection.ts +0 -125
  439. package/src/runtime/task-runner/state-helpers.ts +0 -176
  440. package/src/runtime/task-runner/tail-read.ts +0 -34
  441. package/src/runtime/task-runner.ts +0 -1420
  442. package/src/runtime/team-runner-artifacts.ts +0 -13
  443. package/src/runtime/team-runner.ts +0 -1993
  444. package/src/runtime/tool-output-pruner.ts +0 -333
  445. package/src/runtime/tool-progress.ts +0 -278
  446. package/src/runtime/usage-tracker.ts +0 -73
  447. package/src/runtime/verification-gates.ts +0 -571
  448. package/src/runtime/verification-integrity.ts +0 -105
  449. package/src/runtime/verification-worktree.ts +0 -186
  450. package/src/runtime/worker-heartbeat.ts +0 -25
  451. package/src/runtime/worker-startup.ts +0 -83
  452. package/src/runtime/workflow-state.ts +0 -195
  453. package/src/runtime/workspace-lock.ts +0 -443
  454. package/src/runtime/workspace-tree.ts +0 -312
  455. package/src/runtime/yield-handler.ts +0 -224
  456. package/src/runtime/zombie-scanner.ts +0 -298
  457. package/src/schema/config-schema.ts +0 -327
  458. package/src/schema/team-tool-schema.ts +0 -415
  459. package/src/schema/validation-types.ts +0 -159
  460. package/src/skills/discover-skills.ts +0 -197
  461. package/src/skills/skill-templates.ts +0 -456
  462. package/src/skills/validate.ts +0 -285
  463. package/src/state/active-run-registry.ts +0 -439
  464. package/src/state/artifact-store.ts +0 -176
  465. package/src/state/atomic-write.ts +0 -903
  466. package/src/state/blob-store.ts +0 -308
  467. package/src/state/contracts.ts +0 -158
  468. package/src/state/crew-init.ts +0 -269
  469. package/src/state/decision-ledger.ts +0 -345
  470. package/src/state/event-log-rotation.ts +0 -342
  471. package/src/state/event-log.ts +0 -1311
  472. package/src/state/event-reconstructor.ts +0 -229
  473. package/src/state/gitignore-manager.ts +0 -46
  474. package/src/state/health-store.ts +0 -79
  475. package/src/state/hook-instinct-bridge.ts +0 -94
  476. package/src/state/hook-integrations.ts +0 -51
  477. package/src/state/instinct-store.ts +0 -275
  478. package/src/state/jsonl-writer.ts +0 -107
  479. package/src/state/locks.ts +0 -562
  480. package/src/state/mailbox.ts +0 -916
  481. package/src/state/observation-store.ts +0 -176
  482. package/src/state/run-cache.ts +0 -193
  483. package/src/state/run-graph.ts +0 -183
  484. package/src/state/run-metrics.ts +0 -165
  485. package/src/state/schedule.ts +0 -166
  486. package/src/state/session-state-map.ts +0 -51
  487. package/src/state/state-store.ts +0 -958
  488. package/src/state/task-claims.ts +0 -61
  489. package/src/state/tiered-eval.ts +0 -480
  490. package/src/state/types-eval.ts +0 -58
  491. package/src/state/types.ts +0 -440
  492. package/src/state/usage.ts +0 -138
  493. package/src/state/worker-atomic-writer.ts +0 -194
  494. package/src/subagents/async-entry.ts +0 -1
  495. package/src/subagents/index.ts +0 -3
  496. package/src/subagents/live/control.ts +0 -1
  497. package/src/subagents/live/manager.ts +0 -1
  498. package/src/subagents/live/realtime.ts +0 -1
  499. package/src/subagents/live/session-runtime.ts +0 -1
  500. package/src/subagents/manager.ts +0 -1
  501. package/src/subagents/spawn.ts +0 -1
  502. package/src/teams/discover-teams.ts +0 -187
  503. package/src/teams/team-config.ts +0 -27
  504. package/src/teams/team-serializer.ts +0 -38
  505. package/src/tools/safe-bash-extension.ts +0 -54
  506. package/src/tools/safe-bash.ts +0 -505
  507. package/src/types/diff.d.ts +0 -18
  508. package/src/types/new-api-types.ts +0 -35
  509. package/src/ui/agent-management-overlay.ts +0 -160
  510. package/src/ui/card-colors.ts +0 -139
  511. package/src/ui/crew-footer.ts +0 -102
  512. package/src/ui/crew-select-list.ts +0 -111
  513. package/src/ui/dashboard-panes/agents-pane.ts +0 -164
  514. package/src/ui/dashboard-panes/cancellation-pane.ts +0 -66
  515. package/src/ui/dashboard-panes/capability-pane.ts +0 -77
  516. package/src/ui/dashboard-panes/health-pane.ts +0 -31
  517. package/src/ui/dashboard-panes/mailbox-pane.ts +0 -35
  518. package/src/ui/dashboard-panes/metrics-pane.ts +0 -37
  519. package/src/ui/dashboard-panes/progress-pane.ts +0 -38
  520. package/src/ui/dashboard-panes/transcript-pane.ts +0 -10
  521. package/src/ui/deploy-bundled-themes.ts +0 -71
  522. package/src/ui/dwf-phase-display.ts +0 -151
  523. package/src/ui/dynamic-border.ts +0 -35
  524. package/src/ui/heartbeat-aggregator.ts +0 -74
  525. package/src/ui/key-utils.ts +0 -42
  526. package/src/ui/keybinding-map.ts +0 -248
  527. package/src/ui/layout-primitives.ts +0 -106
  528. package/src/ui/live-conversation-overlay.ts +0 -183
  529. package/src/ui/live-duration.ts +0 -55
  530. package/src/ui/live-run-sidebar.ts +0 -293
  531. package/src/ui/loaders.ts +0 -6
  532. package/src/ui/mascot.ts +0 -449
  533. package/src/ui/overlays/agent-picker-overlay.ts +0 -65
  534. package/src/ui/overlays/confirm-overlay.ts +0 -60
  535. package/src/ui/overlays/help-overlay.ts +0 -177
  536. package/src/ui/overlays/mailbox-compose-overlay.ts +0 -181
  537. package/src/ui/overlays/mailbox-compose-preview.ts +0 -78
  538. package/src/ui/overlays/mailbox-detail-overlay.ts +0 -154
  539. package/src/ui/pi-ui-compat.ts +0 -68
  540. package/src/ui/powerbar-publisher.ts +0 -459
  541. package/src/ui/render-coalescer.ts +0 -101
  542. package/src/ui/render-diff.ts +0 -160
  543. package/src/ui/render-scheduler.ts +0 -278
  544. package/src/ui/run-action-dispatcher.ts +0 -182
  545. package/src/ui/run-dashboard.ts +0 -904
  546. package/src/ui/run-event-bus.ts +0 -381
  547. package/src/ui/run-snapshot-cache.ts +0 -1057
  548. package/src/ui/settings-overlay.ts +0 -1067
  549. package/src/ui/shared-overlay-scheduler.ts +0 -96
  550. package/src/ui/snapshot-types.ts +0 -90
  551. package/src/ui/spinner.ts +0 -17
  552. package/src/ui/status-colors.ts +0 -143
  553. package/src/ui/syntax-highlight.ts +0 -136
  554. package/src/ui/terminal-status.ts +0 -269
  555. package/src/ui/theme-adapter.ts +0 -271
  556. package/src/ui/theme-discovery.ts +0 -199
  557. package/src/ui/tool-progress-formatter.ts +0 -93
  558. package/src/ui/tool-render.ts +0 -453
  559. package/src/ui/tool-renderers/brief-mode.ts +0 -225
  560. package/src/ui/tool-renderers/index.ts +0 -726
  561. package/src/ui/transcript-cache.ts +0 -128
  562. package/src/ui/transcript-entries.ts +0 -256
  563. package/src/ui/transcript-viewer.ts +0 -424
  564. package/src/ui/widget/index.ts +0 -391
  565. package/src/ui/widget/widget-formatters.ts +0 -182
  566. package/src/ui/widget/widget-model.ts +0 -110
  567. package/src/ui/widget/widget-renderer.ts +0 -182
  568. package/src/ui/widget/widget-types.ts +0 -36
  569. package/src/utils/bm25-search.ts +0 -235
  570. package/src/utils/completion-dedupe.ts +0 -63
  571. package/src/utils/conflict-detect.ts +0 -668
  572. package/src/utils/env-allowlist.ts +0 -30
  573. package/src/utils/env-filter.ts +0 -155
  574. package/src/utils/file-coalescer.ts +0 -98
  575. package/src/utils/fingerprint.ts +0 -180
  576. package/src/utils/frontmatter.ts +0 -70
  577. package/src/utils/fs-watch.ts +0 -37
  578. package/src/utils/gh-protocol.ts +0 -556
  579. package/src/utils/git.ts +0 -260
  580. package/src/utils/guards.ts +0 -107
  581. package/src/utils/ids.ts +0 -24
  582. package/src/utils/incremental-reader.ts +0 -220
  583. package/src/utils/internal-error.ts +0 -10
  584. package/src/utils/names.ts +0 -36
  585. package/src/utils/ndjson.ts +0 -115
  586. package/src/utils/paths.ts +0 -227
  587. package/src/utils/project-detector.ts +0 -160
  588. package/src/utils/redaction.ts +0 -347
  589. package/src/utils/resolve-shell.ts +0 -36
  590. package/src/utils/run-watcher-registry.ts +0 -152
  591. package/src/utils/safe-paths.ts +0 -383
  592. package/src/utils/scan-cache.ts +0 -144
  593. package/src/utils/session-utils.ts +0 -110
  594. package/src/utils/sleep.ts +0 -54
  595. package/src/utils/socket-path.ts +0 -127
  596. package/src/utils/sse-parser.ts +0 -129
  597. package/src/utils/task-name-generator.ts +0 -337
  598. package/src/utils/timings.ts +0 -33
  599. package/src/utils/token-counter.ts +0 -200
  600. package/src/utils/visual.ts +0 -207
  601. package/src/workflows/cost-estimator.ts +0 -34
  602. package/src/workflows/discover-workflows.ts +0 -290
  603. package/src/workflows/intermediate-store.ts +0 -166
  604. package/src/workflows/preflight-validator.ts +0 -168
  605. package/src/workflows/topology-analyzer.ts +0 -203
  606. package/src/workflows/validate-workflow.ts +0 -50
  607. package/src/workflows/workflow-config.ts +0 -70
  608. package/src/workflows/workflow-serializer.ts +0 -32
  609. package/src/worktree/branch-freshness.ts +0 -112
  610. package/src/worktree/cleanup.ts +0 -341
  611. package/src/worktree/worktree-manager.ts +0 -1200
@@ -1,961 +0,0 @@
1
- # Historical Bug Reports (v0.2.x)
2
-
3
- > **Current version: v0.9.0** — See [CHANGELOG.md](../CHANGELOG.md) for all bug fixes.
4
- > This page tracks historical bugs from v0.2.x. All listed bugs are fixed.
5
-
6
- ---
7
-
8
- # pi-crew v0.2.20 — Bug Report & Fixes
9
-
10
- **Date:** 2026-05-19
11
- **Session:** Comprehensive integration test + root cause analysis
12
- **Environment:** linux/x64, Node v22.22.0, Pi CLI v0.75.3, pi-crew v0.2.20
13
- **Status:** ✅ 14/14 bugs fixed (commits `de9e8b4` and `5dc794e`)
14
-
15
- > **All bugs fixed ✅** — Source code verified. See [pi-crew-test-final.md](pi-crew-test-final.md) for end-to-end test results.
16
-
17
- ---
18
-
19
- ## Bug #1: Background workers "heartbeat dead" — actually a MiniMax 429 Rate Limit
20
-
21
- | Field | Value |
22
- |---|---|
23
- | **Severity** | 🔴 HIGH |
24
- | **Status** | ✅ Fixed — 429 now retries with fallback models instead of blocking 300s |
25
- | **Affected** | All background/async workers |
26
- | **Symptom** | Workers time out after 300s with "heartbeat dead", zero output |
27
-
28
- ### Description
29
-
30
- When running `team action='run'` with `async=true` or `Agent(run_in_background=true)`, workers spawn successfully (PID exists) but **time out after 300s** with a generic error:
31
- ```
32
- worker.response_timeout: No output for 300000ms
33
- crew.task.heartbeat_dead: Task 01_assess heartbeat dead.
34
- ```
35
-
36
- ### Root cause
37
-
38
- **Fixed.** Previously the 429 rate limit was not retried because:
39
- 1. `RETRYABLE_MODEL_FAILURE_PATTERNS` had `/\b429\b/` but MiniMax returns `rate_limit_error: usage limit exceeded` (no clear "429" number)
40
- 2. 429 was fast-failed in `child-pi.ts onJsonEvent` instead of letting the task-runner handle retry with fallback
41
-
42
- ### Fix applied
43
-
44
- 1. **model-fallback.ts**: Added `/rate_limit_error/i` to `RETRYABLE_MODEL_FAILURE_PATTERNS` to correctly identify the MiniMax rate limit error
45
- 2. **model-fallback.ts**: Changed `/\b429\b/` → `/rate.?limit/i` to match more formats
46
- 3. **child-pi.ts**: Removed the 429 fast-fail — let the task-runner handle retry with the model fallback chain
47
-
48
- ### Model fallback chain
49
-
50
- When the main model gets a 429:
51
- 1. Fall back to `fallbackModels` (if configured)
52
- 2. Fall back to other available models in the system
53
- 3. If there is no fallback and retries are exhausted → fail with the correct error message
54
-
55
- **Recommended config:** Add `fallbackModels` to the agent config to have more options when the main model is rate-limited.
56
-
57
- ---
58
-
59
- ## Bug #2: child-pi.ts does not detect 429 rate limit error — reports wrong "heartbeat dead"
60
-
61
- | Field | Value |
62
- |---|---|
63
- | **Severity** | 🔴 HIGH |
64
- | **Status** | New — discovered while debugging Bug #1 |
65
- | **Affected** | All child Pi workers |
66
- | **Symptom** | Worker reports generic "No output for 300000ms" instead of "Provider rate limit: 429" |
67
-
68
- ### Description
69
-
70
- The Pi CLI outputs JSON events for 429 errors very clearly:
71
- ```json
72
- {"type":"turn_end","message":{"stopReason":"error","errorMessage":"429 {\"type\":\"error\",\"error\":{\"type\":\"rate_limit_error\"...}}"}}
73
- ```
74
-
75
- But `child-pi.ts` **does not parse error events** — it only cares about:
76
- - `isFinalAssistantEvent()` — to trigger the final drain
77
- - `turn_end` — to count turns for turn limiting
78
-
79
- Result: child-pi sees output (JSON events), **restarts the heartbeat timer**, but **does not recognize it as an error**. Pi blocks after 3 retries → heartbeat times out at 300s → generic error message.
80
-
81
- ### Code location
82
-
83
- `/home/bom/source/my_pi/pi-crew/src/runtime/child-pi.ts`, line ~394:
84
- ```typescript
85
- onJsonEvent: (event) => {
86
- restartNoResponseTimer();
87
- // Turn-count-based steering: only counts turns, does NOT check errors
88
- if (event && typeof event === "object" && !Array.isArray(event)) {
89
- const obj = event as Record<string, unknown>;
90
- if (obj.type === "turn_end") {
91
- turnCount += 1;
92
- // ... turn limit logic only ...
93
- }
94
- }
95
- // MISSING: detect provider errors (429, auth, etc.)
96
- }
97
- ```
98
-
99
- ### Fix
100
-
101
- Add provider error detection in `onJsonEvent`:
102
- ```typescript
103
- let providerError: string | undefined;
104
-
105
- // In onJsonEvent:
106
- if (obj.type === "turn_end" && obj.message?.stopReason === "error") {
107
- const errMsg = obj.message?.errorMessage || "";
108
- if (errMsg && !providerError) providerError = errMsg;
109
- // Fast-fail on rate limit — don't wait 300s
110
- if (/429|rate.?limit/i.test(errMsg)) {
111
- settle({ exitCode: 1, stdout, stderr: `Provider rate limit: ${errMsg.slice(0, 200)}` });
112
- }
113
- }
114
- ```
115
-
116
- ### Impact
117
-
118
- This fix changes the error message from:
119
- ```
120
- ❌ "Child Pi produced no new output for 300000ms; process was terminated as unresponsive."
121
- ```
122
- to:
123
- ```
124
- ✅ "Provider rate limit: 429 rate_limit_error: usage limit exceeded, resets at 2026-05-19T05:00:00Z"
125
- ```
126
-
127
- And it **fails fast** instead of waiting 300s.
128
-
129
- ---
130
-
131
- ## Bug #3: background.log is useless — does not capture worker output
132
-
133
- | Field | Value |
134
- |---|---|
135
- | **Severity** | 🟠 MEDIUM |
136
- | **Status** | New — discovered while debugging Bug #1 |
137
- | **Affected** | Debugging experience for all background runs |
138
- | **Symptom** | background.log contains only 1 line: `[pi-crew] background loader=jiti` |
139
-
140
- ### Description
141
-
142
- When a background worker fails, the log file at `.crew/state/runs/<id>/background.log` contains only:
143
- ```
144
- [pi-crew] background loader=jiti
145
- ```
146
-
147
- Missing:
148
- - Worker stdout/stderr
149
- - Error messages
150
- - Provider responses
151
- - Exit codes
152
-
153
- ### Cause
154
-
155
- `async-runner.ts` line 130-145:
156
- ```typescript
157
- const logFd = fs.openSync(logPath, "a");
158
- // ...
159
- const child = spawn(process.execPath, command.args, buildBackgroundSpawnOptions(manifest, logFd));
160
- ```
161
-
162
- `buildBackgroundSpawnOptions` line 123-127:
163
- ```typescript
164
- return {
165
- cwd: manifest.cwd,
166
- detached: true,
167
- stdio: ["ignore", logFd, logFd], // stdout+stderr → background.log
168
- // ...
169
- };
170
- ```
171
-
172
- **The stdout/stderr of the background-runner** is written to background.log. But **child Pi workers** (spawned by the background-runner via child-pi.ts) **output to child-pi's pipe**, NOT to background.log.
173
-
174
- Flow:
175
- ```
176
- background-runner.ts (stdout→logFd, stderr→logFd)
177
- → loader=jiti → writes to log ✅
178
- → executeTeamRun()
179
- → child-pi.ts spawns child Pi (stdout→pipe, stderr→pipe)
180
- → Pi output → child-pi.ts captures → DOES NOT WRITE TO background.log ❌
181
- ```
182
-
183
- ### Fix
184
-
185
- 1. **Option A:** In `child-pi.ts` or `team-runner.ts`, write worker output events to background.log
186
- 2. **Option B:** Add event log entries for provider errors (there is an event log, but not detailed enough)
187
- 3. **Option C:** Background-runner tees output to a log file
188
-
189
- ### Key file
190
-
191
- ```
192
- pi-crew/src/runtime/async-runner.ts — buildBackgroundSpawnOptions(), spawnBackgroundTeamRun()
193
- ```
194
-
195
- ---
196
-
197
- ## Bug #4: worker-startup.ts missing "rate_limited" classification
198
-
199
- | Field | Value |
200
- |---|---|
201
- | **Severity** | 🟡 LOW |
202
- | **Status** | New — discovered while debugging Bug #1 |
203
- | **Affected** | Error classification and reporting |
204
- | **Symptom** | 429 errors classified as "unknown" instead of "rate_limited" |
205
-
206
- ### Description
207
-
208
- `worker-startup.ts` has the `StartupFailureClassification` type:
209
- ```typescript
210
- export type StartupFailureClassification =
211
- | "trust_required"
212
- | "prompt_misdelivery"
213
- | "prompt_acceptance_timeout"
214
- | "transport_dead"
215
- | "worker_crashed"
216
- | "unknown";
217
- ```
218
-
219
- Missing `"rate_limited"` and `"provider_error"`. Result: 429 errors are classified as `"unknown"`.
220
-
221
- ### Fix
222
-
223
- Add to the type and `classifyStartupFailure` function:
224
- ```typescript
225
- export type StartupFailureClassification =
226
- | "trust_required"
227
- | "prompt_misdelivery"
228
- | "prompt_acceptance_timeout"
229
- | "transport_dead"
230
- | "worker_crashed"
231
- | "rate_limited" // NEW
232
- | "provider_error" // NEW
233
- | "unknown";
234
-
235
- // In classifyStartupFailure:
236
- if (evidence.stderrPreview && /429|rate.?limit/i.test(evidence.stderrPreview)) return "rate_limited";
237
- if (evidence.stderrPreview && /5\d{2}|server.?error|internal.?error/i.test(evidence.stderrPreview)) return "provider_error";
238
- ```
239
-
240
- ### Key file
241
-
242
- ```
243
- pi-crew/src/runtime/worker-startup.ts — StartupFailureClassification, classifyStartupFailure()
244
- ```
245
-
246
- ---
247
-
248
- ## Bug #5: Stale heartbeat notifications after prune
249
-
250
- | Field | Value |
251
- |---|---|
252
- | **Severity** | 🟡 LOW (cosmetic) |
253
- | **Status** | Confirmed |
254
- | **Affected** | User experience |
255
- | **Symptom** | "Task heartbeat dead" notifications for already-removed runs |
256
-
257
- ### Description
258
-
259
- After running `team prune --keep=0 --confirm=true`, the background watcher still emits notifications for pruned runs:
260
-
261
- ```
262
- → team prune: Removed 9 runs
263
- → Notification: "agent_mpc423rq_1 heartbeat dead" (run not found)
264
- → Notification: "agent_mpc423rv_2 heartbeat dead" (run not found)
265
- → Notification: "agent_mpc423rw_3 heartbeat dead" (run not found)
266
- → Notification: "agent_mpc423rw_4 heartbeat dead" (run not found)
267
- ... (6+ stale notifications)
268
- ```
269
-
270
- Each notification triggers `get_subagent_result` → returns "not found".
271
-
272
- ### Cause
273
-
274
- The background watcher maintains a worker health-check queue. When runs are pruned:
275
- 1. The watcher does not deregister immediately
276
- 2. Notifications already in the queue still emit
277
- 3. The notifications arrive one by one, a few seconds apart
278
-
279
- ### Impact
280
-
281
- - Confusing for the user: seeing "heartbeat dead" for runs that no longer exist
282
- - Wasted context: each notification triggers 1 tool call to verify
283
-
284
- ### Fix
285
-
286
- The background watcher should check run existence before emitting:
287
- ```typescript
288
- // Before emitting heartbeat_dead:
289
- if (!runExists(runId)) {
290
- deregisterWorker(workerId); // Silent cleanup
291
- return;
292
- }
293
- ```
294
-
295
- ### Key files
296
-
297
- ```
298
- pi-crew/src/runtime/worker-heartbeat.ts — isWorkerHeartbeatStale()
299
- pi-crew/src/runtime/background-runner.ts — heartbeat monitoring loop
300
- ```
301
-
302
- ---
303
-
304
- # pi-crew v0.2.20 — Bug Report
305
-
306
- **Date:** 2026-05-19
307
- **Session:** Comprehensive integration test + root cause analysis
308
- **Environment:** linux/x64, Node v22.22.0, Pi CLI v0.75.3, pi-crew v0.2.20
309
-
310
- ---
311
-
312
- ## Bug #1: Background workers "heartbeat dead" — actually a MiniMax 429 Rate Limit
313
-
314
- | Field | Value |
315
- |---|---|
316
- | **Severity** | 🔴 HIGH |
317
- | **Status** | ✅ Fixed — 429 now retries with fallback models instead of blocking 300s |
318
- | **Affected** | All background/async workers |
319
- | **Symptom** | Workers time out after 300s with "heartbeat dead", zero output |
320
-
321
- ### Description
322
-
323
- When running `team action='run'` with `async=true` or `Agent(run_in_background=true)`, workers spawn successfully (PID exists) but **time out after 300s** with a generic error:
324
- ```
325
- worker.response_timeout: No output for 300000ms
326
- crew.task.heartbeat_dead: Task 01_assess heartbeat dead.
327
- ```
328
-
329
- ### Root cause
330
-
331
- **Fixed.** Previously the 429 rate limit was not retried because:
332
- 1. `RETRYABLE_MODEL_FAILURE_PATTERNS` had `/\b429\b/` but MiniMax returns `rate_limit_error: usage limit exceeded` (no clear "429" number)
333
- 2. 429 was fast-failed in `child-pi.ts onJsonEvent` instead of letting the task-runner handle retry with fallback
334
-
335
- ### Fix applied
336
-
337
- 1. **model-fallback.ts**: Added `/rate_limit_error/i` to `RETRYABLE_MODEL_FAILURE_PATTERNS` to correctly identify the MiniMax rate limit error
338
- 2. **model-fallback.ts**: Changed `/\b429\b/` → `/rate.?limit/i` to match more formats
339
- 3. **child-pi.ts**: Removed the 429 fast-fail — let the task-runner handle retry with the model fallback chain
340
-
341
- ### Model fallback chain
342
-
343
- When the main model gets a 429:
344
- 1. Fall back to `fallbackModels` (if configured)
345
- 2. Fall back to other available models in the system
346
- 3. If there is no fallback and retries are exhausted → fail with the correct error message
347
-
348
- **Recommended config:** Add `fallbackModels` to the agent config to have more options when the main model is rate-limited.
349
-
350
- ---
351
-
352
- ## Bug #2: child-pi.ts does not detect 429 rate limit error — reports wrong "heartbeat dead"
353
-
354
- | Field | Value |
355
- |---|---|
356
- | **Severity** | 🔴 HIGH |
357
- | **Status** | New — discovered while debugging Bug #1 |
358
- | **Affected** | All child Pi workers |
359
- | **Symptom** | Worker reports generic "No output for 300000ms" instead of "Provider rate limit: 429" |
360
-
361
- ### Description
362
-
363
- The Pi CLI outputs JSON events for 429 errors very clearly:
364
- ```json
365
- {"type":"turn_end","message":{"stopReason":"error","errorMessage":"429 {\"type\":\"error\",\"error\":{\"type\":\"rate_limit_error\"...}}"}}
366
- ```
367
-
368
- But `child-pi.ts` **does not parse error events** — it only cares about:
369
- - `isFinalAssistantEvent()` — to trigger the final drain
370
- - `turn_end` — to count turns for turn limiting
371
-
372
- Result: child-pi sees output (JSON events), **restarts the heartbeat timer**, but **does not recognize it as an error**. Pi blocks after 3 retries → heartbeat times out at 300s → generic error message.
373
-
374
- ### Code location
375
-
376
- `/home/bom/source/my_pi/pi-crew/src/runtime/child-pi.ts`, line ~394:
377
- ```typescript
378
- onJsonEvent: (event) => {
379
- restartNoResponseTimer();
380
- // Turn-count-based steering: only counts turns, does NOT check errors
381
- if (event && typeof event === "object" && !Array.isArray(event)) {
382
- const obj = event as Record<string, unknown>;
383
- if (obj.type === "turn_end") {
384
- turnCount += 1;
385
- // ... turn limit logic only ...
386
- }
387
- }
388
- // MISSING: detect provider errors (429, auth, etc.)
389
- }
390
- ```
391
-
392
- ### Fix
393
-
394
- Add provider error detection in `onJsonEvent`:
395
- ```typescript
396
- let providerError: string | undefined;
397
-
398
- // In onJsonEvent:
399
- if (obj.type === "turn_end" && obj.message?.stopReason === "error") {
400
- const errMsg = obj.message?.errorMessage || "";
401
- if (errMsg && !providerError) providerError = errMsg;
402
- // Fast-fail on rate limit — don't wait 300s
403
- if (/429|rate.?limit/i.test(errMsg)) {
404
- settle({ exitCode: 1, stdout, stderr: `Provider rate limit: ${errMsg.slice(0, 200)}` });
405
- }
406
- }
407
- ```
408
-
409
- ### Impact
410
-
411
- This fix changes the error message from:
412
- ```
413
- ❌ "Child Pi produced no new output for 300000ms; process was terminated as unresponsive."
414
- ```
415
- to:
416
- ```
417
- ✅ "Provider rate limit: 429 rate_limit_error: usage limit exceeded, resets at 2026-05-19T05:00:00Z"
418
- ```
419
-
420
- And it **fails fast** instead of waiting 300s.
421
-
422
- ---
423
-
424
- ## Bug #3: background.log is useless — does not capture worker output
425
-
426
- | Field | Value |
427
- |---|---|
428
- | **Severity** | 🟠 MEDIUM |
429
- | **Status** | New — discovered while debugging Bug #1 |
430
- | **Affected** | Debugging experience for all background runs |
431
- | **Symptom** | background.log contains only 1 line: `[pi-crew] background loader=jiti` |
432
-
433
- ### Description
434
-
435
- When a background worker fails, the log file at `.crew/state/runs/<id>/background.log` contains only:
436
- ```
437
- [pi-crew] background loader=jiti
438
- ```
439
-
440
- Missing:
441
- - Worker stdout/stderr
442
- - Error messages
443
- - Provider responses
444
- - Exit codes
445
-
446
- ### Cause
447
-
448
- `async-runner.ts` line 130-145:
449
- ```typescript
450
- const logFd = fs.openSync(logPath, "a");
451
- // ...
452
- const child = spawn(process.execPath, command.args, buildBackgroundSpawnOptions(manifest, logFd));
453
- ```
454
-
455
- `buildBackgroundSpawnOptions` line 123-127:
456
- ```typescript
457
- return {
458
- cwd: manifest.cwd,
459
- detached: true,
460
- stdio: ["ignore", logFd, logFd], // stdout+stderr → background.log
461
- // ...
462
- };
463
- ```
464
-
465
- **The stdout/stderr of the background-runner** is written to background.log. But **child Pi workers** (spawned by the background-runner via child-pi.ts) **output to child-pi's pipe**, NOT to background.log.
466
-
467
- Flow:
468
- ```
469
- background-runner.ts (stdout→logFd, stderr→logFd)
470
- → loader=jiti → writes to log ✅
471
- → executeTeamRun()
472
- → child-pi.ts spawns child Pi (stdout→pipe, stderr→pipe)
473
- → Pi output → child-pi.ts captures → DOES NOT WRITE TO background.log ❌
474
- ```
475
-
476
- ### Fix
477
-
478
- 1. **Option A:** In `child-pi.ts` or `team-runner.ts`, write worker output events to background.log
479
- 2. **Option B:** Add event log entries for provider errors (there is an event log, but not detailed enough)
480
- 3. **Option C:** Background-runner tees output to a log file
481
-
482
- ### Key file
483
-
484
- ```
485
- pi-crew/src/runtime/async-runner.ts — buildBackgroundSpawnOptions(), spawnBackgroundTeamRun()
486
- ```
487
-
488
- ---
489
-
490
- ## Bug #4: worker-startup.ts missing "rate_limited" classification
491
-
492
- | Field | Value |
493
- |---|---|
494
- | **Severity** | 🟡 LOW |
495
- | **Status** | New — discovered while debugging Bug #1 |
496
- | **Affected** | Error classification and reporting |
497
- | **Symptom** | 429 errors classified as "unknown" instead of "rate_limited" |
498
-
499
- ### Description
500
-
501
- `worker-startup.ts` has the `StartupFailureClassification` type:
502
- ```typescript
503
- export type StartupFailureClassification =
504
- | "trust_required"
505
- | "prompt_misdelivery"
506
- | "prompt_acceptance_timeout"
507
- | "transport_dead"
508
- | "worker_crashed"
509
- | "unknown";
510
- ```
511
-
512
- Missing `"rate_limited"` and `"provider_error"`. Result: 429 errors are classified as `"unknown"`.
513
-
514
- ### Fix
515
-
516
- Add to the type and `classifyStartupFailure` function:
517
- ```typescript
518
- export type StartupFailureClassification =
519
- | "trust_required"
520
- | "prompt_misdelivery"
521
- | "prompt_acceptance_timeout"
522
- | "transport_dead"
523
- | "worker_crashed"
524
- | "rate_limited" // NEW
525
- | "provider_error" // NEW
526
- | "unknown";
527
-
528
- // In classifyStartupFailure:
529
- if (evidence.stderrPreview && /429|rate.?limit/i.test(evidence.stderrPreview)) return "rate_limited";
530
- if (evidence.stderrPreview && /5\d{2}|server.?error|internal.?error/i.test(evidence.stderrPreview)) return "provider_error";
531
- ```
532
-
533
- ### Key file
534
-
535
- ```
536
- pi-crew/src/runtime/worker-startup.ts — StartupFailureClassification, classifyStartupFailure()
537
- ```
538
-
539
- ---
540
-
541
- ## Bug #5: Stale heartbeat notifications after prune
542
-
543
- | Field | Value |
544
- |---|---|
545
- | **Severity** | 🟡 LOW (cosmetic) |
546
- | **Status** | Confirmed |
547
- | **Affected** | User experience |
548
- | **Symptom** | "Task heartbeat dead" notifications for already-removed runs |
549
-
550
- ### Description
551
-
552
- After running `team prune --keep=0 --confirm=true`, the background watcher still emits notifications for pruned runs:
553
-
554
- ```
555
- → team prune: Removed 9 runs
556
- → Notification: "agent_mpc423rq_1 heartbeat dead" (run not found)
557
- → Notification: "agent_mpc423rv_2 heartbeat dead" (run not found)
558
- → Notification: "agent_mpc423rw_3 heartbeat dead" (run not found)
559
- → Notification: "agent_mpc423rw_4 heartbeat dead" (run not found)
560
- ... (6+ stale notifications)
561
- ```
562
-
563
- Each notification triggers `get_subagent_result` → returns "not found".
564
-
565
- ### Cause
566
-
567
- The background watcher maintains a worker health-check queue. When runs are pruned:
568
- 1. The watcher does not deregister immediately
569
- 2. Notifications already in the queue still emit
570
- 3. The notifications arrive one by one, a few seconds apart
571
-
572
- ### Impact
573
-
574
- - Confusing for the user: seeing "heartbeat dead" for runs that no longer exist
575
- - Wasted context: each notification triggers 1 tool call to verify
576
-
577
- ### Fix
578
-
579
- The background watcher should check run existence before emitting:
580
- ```typescript
581
- // Before emitting heartbeat_dead:
582
- if (!runExists(runId)) {
583
- deregisterWorker(workerId); // Silent cleanup
584
- return;
585
- }
586
- ```
587
-
588
- ### Key files
589
-
590
- ```
591
- pi-crew/src/runtime/worker-heartbeat.ts — isWorkerHeartbeatStale()
592
- pi-crew/src/runtime/background-runner.ts — heartbeat monitoring loop
593
- ```
594
-
595
- ---
596
-
597
- # pi-crew v0.2.20 — Bug Report
598
-
599
- **Date:** 2026-05-19
600
- **Session:** Comprehensive integration test + root cause analysis
601
- **Environment:** linux/x64, Node v22.22.0, Pi CLI v0.75.3, pi-crew v0.2.20
602
-
603
- ---
604
-
605
- ## Bug #1: Background workers "heartbeat dead" — actually a MiniMax 429 Rate Limit
606
-
607
- | Field | Value |
608
- |---|---|
609
- | **Severity** | 🔴 HIGH |
610
- | **Status** | ✅ Fixed — 429 now retries with fallback models instead of blocking 300s |
611
- | **Affected** | All background/async workers |
612
- | **Symptom** | Workers time out after 300s with "heartbeat dead", zero output |
613
-
614
- ### Description
615
-
616
- When running `team action='run'` with `async=true` or `Agent(run_in_background=true)`, workers spawn successfully (PID exists) but **time out after 300s** with a generic error:
617
- ```
618
- worker.response_timeout: No output for 300000ms
619
- crew.task.heartbeat_dead: Task 01_assess heartbeat dead.
620
- ```
621
-
622
- ### Root cause
623
-
624
- **Fixed.** Previously the 429 rate limit was not retried because:
625
- 1. `RETRYABLE_MODEL_FAILURE_PATTERNS` had `/\b429\b/` but MiniMax returns `rate_limit_error: usage limit exceeded` (no clear "429" number)
626
- 2. 429 was fast-failed in `child-pi.ts onJsonEvent` instead of letting the task-runner handle retry with fallback
627
-
628
- ### Fix applied
629
-
630
- 1. **model-fallback.ts**: Added `/rate_limit_error/i` to `RETRYABLE_MODEL_FAILURE_PATTERNS` to correctly identify the MiniMax rate limit error
631
- 2. **model-fallback.ts**: Changed `/\b429\b/` → `/rate.?limit/i` to match more formats
632
- 3. **child-pi.ts**: Removed the 429 fast-fail — let the task-runner handle retry with the model fallback chain
633
-
634
- ### Model fallback chain
635
-
636
- When the main model gets a 429:
637
- 1. Fall back to `fallbackModels` (if configured)
638
- 2. Fall back to other available models in the system
639
- 3. If there is no fallback and retries are exhausted → fail with the correct error message
640
-
641
- **Recommended config:** Add `fallbackModels` to the agent config to have more options when the main model is rate-limited.
642
-
643
- ---
644
-
645
- ## Bug #2: child-pi.ts does not detect 429 rate limit error — reports wrong "heartbeat dead"
646
-
647
- | Field | Value |
648
- |---|---|
649
- | **Severity** | 🔴 HIGH |
650
- | **Status** | New — discovered while debugging Bug #1 |
651
- | **Affected** | All child Pi workers |
652
- | **Symptom** | Worker reports generic "No output for 300000ms" instead of "Provider rate limit: 429" |
653
-
654
- ### Description
655
-
656
- The Pi CLI outputs JSON events for 429 errors very clearly:
657
- ```json
658
- {"type":"turn_end","message":{"stopReason":"error","errorMessage":"429 {\"type\":\"error\",\"error\":{\"type\":\"rate_limit_error\"...}}"}}
659
- ```
660
-
661
- But `child-pi.ts` **does not parse error events** — it only cares about:
662
- - `isFinalAssistantEvent()` — to trigger the final drain
663
- - `turn_end` — to count turns for turn limiting
664
-
665
- Result: child-pi sees output (JSON events), **restarts the heartbeat timer**, but **does not recognize it as an error**. Pi blocks after 3 retries → heartbeat times out at 300s → generic error message.
666
-
667
- ### Code location
668
-
669
- `/home/bom/source/my_pi/pi-crew/src/runtime/child-pi.ts`, line ~394:
670
- ```typescript
671
- onJsonEvent: (event) => {
672
- restartNoResponseTimer();
673
- // Turn-count-based steering: only counts turns, does NOT check errors
674
- if (event && typeof event === "object" && !Array.isArray(event)) {
675
- const obj = event as Record<string, unknown>;
676
- if (obj.type === "turn_end") {
677
- turnCount += 1;
678
- // ... turn limit logic only ...
679
- }
680
- }
681
- // MISSING: detect provider errors (429, auth, etc.)
682
- }
683
- ```
684
-
685
- ### Fix
686
-
687
- Add provider error detection in `onJsonEvent`:
688
- ```typescript
689
- let providerError: string | undefined;
690
-
691
- // In onJsonEvent:
692
- if (obj.type === "turn_end" && obj.message?.stopReason === "error") {
693
- const errMsg = obj.message?.errorMessage || "";
694
- if (errMsg && !providerError) providerError = errMsg;
695
- // Fast-fail on rate limit — don't wait 300s
696
- if (/429|rate.?limit/i.test(errMsg)) {
697
- settle({ exitCode: 1, stdout, stderr: `Provider rate limit: ${errMsg.slice(0, 200)}` });
698
- }
699
- }
700
- ```
701
-
702
- ### Impact
703
-
704
- This fix changes the error message from:
705
- ```
706
- ❌ "Child Pi produced no new output for 300000ms; process was terminated as unresponsive."
707
- ```
708
- to:
709
- ```
710
- ✅ "Provider rate limit: 429 rate_limit_error: usage limit exceeded, resets at 2026-05-19T05:00:00Z"
711
- ```
712
-
713
- And it **fails fast** instead of waiting 300s.
714
-
715
- ---
716
-
717
- ## Bug #3: background.log is useless — does not capture worker output
718
-
719
- | Field | Value |
720
- |---|---|
721
- | **Severity** | 🟠 MEDIUM |
722
- | **Status** | New — discovered while debugging Bug #1 |
723
- | **Affected** | Debugging experience for all background runs |
724
- | **Symptom** | background.log contains only 1 line: `[pi-crew] background loader=jiti` |
725
-
726
- ### Description
727
-
728
- When a background worker fails, the log file at `.crew/state/runs/<id>/background.log` contains only:
729
- ```
730
- [pi-crew] background loader=jiti
731
- ```
732
-
733
- Missing:
734
- - Worker stdout/stderr
735
- - Error messages
736
- - Provider responses
737
- - Exit codes
738
-
739
- ### Cause
740
-
741
- `async-runner.ts` line 130-145:
742
- ```typescript
743
- const logFd = fs.openSync(logPath, "a");
744
- // ...
745
- const child = spawn(process.execPath, command.args, buildBackgroundSpawnOptions(manifest, logFd));
746
- ```
747
-
748
- `buildBackgroundSpawnOptions` line 123-127:
749
- ```typescript
750
- return {
751
- cwd: manifest.cwd,
752
- detached: true,
753
- stdio: ["ignore", logFd, logFd], // stdout+stderr → background.log
754
- // ...
755
- };
756
- ```
757
-
758
- **The stdout/stderr of the background-runner** is written to background.log. But **child Pi workers** (spawned by the background-runner via child-pi.ts) **output to child-pi's pipe**, NOT to background.log.
759
-
760
- Flow:
761
- ```
762
- background-runner.ts (stdout→logFd, stderr→logFd)
763
- → loader=jiti → writes to log ✅
764
- → executeTeamRun()
765
- → child-pi.ts spawns child Pi (stdout→pipe, stderr→pipe)
766
- → Pi output → child-pi.ts captures → DOES NOT WRITE TO background.log ❌
767
- ```
768
-
769
- ### Fix
770
-
771
- 1. **Option A:** In `child-pi.ts` or `team-runner.ts`, write worker output events to background.log
772
- 2. **Option B:** Add event log entries for provider errors (there is an event log, but not detailed enough)
773
- 3. **Option C:** Background-runner tees output to a log file
774
-
775
- ### Key file
776
-
777
- ```
778
- pi-crew/src/runtime/async-runner.ts — buildBackgroundSpawnOptions(), spawnBackgroundTeamRun()
779
- ```
780
-
781
- ---
782
-
783
- ## Bug #4: worker-startup.ts missing "rate_limited" classification
784
-
785
- | Field | Value |
786
- |---|---|
787
- | **Severity** | 🟡 LOW |
788
- | **Status** | New — discovered while debugging Bug #1 |
789
- | **Affected** | Error classification and reporting |
790
- | **Symptom** | 429 errors classified as "unknown" instead of "rate_limited" |
791
-
792
- ### Description
793
-
794
- `worker-startup.ts` has the `StartupFailureClassification` type:
795
- ```typescript
796
- export type StartupFailureClassification =
797
- | "trust_required"
798
- | "prompt_misdelivery"
799
- | "prompt_acceptance_timeout"
800
- | "transport_dead"
801
- | "worker_crashed"
802
- | "unknown";
803
- ```
804
-
805
- Missing `"rate_limited"` and `"provider_error"`. Result: 429 errors are classified as `"unknown"`.
806
-
807
- ### Fix
808
-
809
- Add to the type and `classifyStartupFailure` function:
810
- ```typescript
811
- export type StartupFailureClassification =
812
- | "trust_required"
813
- | "prompt_misdelivery"
814
- | "prompt_acceptance_timeout"
815
- | "transport_dead"
816
- | "worker_crashed"
817
- | "rate_limited" // NEW
818
- | "provider_error" // NEW
819
- | "unknown";
820
-
821
- // In classifyStartupFailure:
822
- if (evidence.stderrPreview && /429|rate.?limit/i.test(evidence.stderrPreview)) return "rate_limited";
823
- if (evidence.stderrPreview && /5\d{2}|server.?error|internal.?error/i.test(evidence.stderrPreview)) return "provider_error";
824
- ```
825
-
826
- ### Key file
827
-
828
- ```
829
- pi-crew/src/runtime/worker-startup.ts — StartupFailureClassification, classifyStartupFailure()
830
- ```
831
-
832
- ---
833
-
834
- ## Bug #5: Stale heartbeat notifications after prune
835
-
836
- | Field | Value |
837
- |---|---|
838
- | **Severity** | 🟡 LOW (cosmetic) |
839
- | **Status** | Confirmed |
840
- | **Affected** | User experience |
841
- | **Symptom** | "Task heartbeat dead" notifications for already-removed runs |
842
-
843
- ### Description
844
-
845
- After running `team prune --keep=0 --confirm=true`, the background watcher still emits notifications for pruned runs:
846
-
847
- ```
848
- → team prune: Removed 9 runs
849
- → Notification: "agent_mpc423rq_1 heartbeat dead" (run not found)
850
- → Notification: "agent_mpc423rv_2 heartbeat dead" (run not found)
851
- → Notification: "agent_mpc423rw_3 heartbeat dead" (run not found)
852
- → Notification: "agent_mpc423rw_4 heartbeat dead" (run not found)
853
- ... (6+ stale notifications)
854
- ```
855
-
856
- Each notification triggers `get_subagent_result` → returns "not found".
857
-
858
- ### Cause
859
-
860
- The background watcher maintains a worker health-check queue. When runs are pruned:
861
- 1. The watcher does not deregister immediately
862
- 2. Notifications already in the queue still emit
863
- 3. The notifications arrive one by one, a few seconds apart
864
-
865
- ### Impact
866
-
867
- - Confusing for the user: seeing "heartbeat dead" for runs that no longer exist
868
- - Wasted context: each notification triggers 1 tool call to verify
869
-
870
- ### Fix
871
-
872
- The background watcher should check run existence before emitting:
873
- ```typescript
874
- // Before emitting heartbeat_dead:
875
- if (!runExists(runId)) {
876
- deregisterWorker(workerId); // Silent cleanup
877
- return;
878
- }
879
- ```
880
-
881
- ### Key files
882
-
883
- ```
884
- pi-crew/src/runtime/worker-heartbeat.ts — isWorkerHeartbeatStale()
885
- pi-crew/src/runtime/background-runner.ts — heartbeat monitoring loop
886
- ```
887
-
888
- ---
889
-
890
- ## Bug #6: Live-session run cancelled mid-execution
891
-
892
- | Field | Value |
893
- |---|---|
894
- | **Severity** | 🟠 MEDIUM |
895
- | **Status** | ✅ Confirmed — no code fix needed; documented as a user workflow constraint |
896
- | **Affected** | Foreground team runs |
897
- | **Symptom** | Run cancelled after the explore phase completes, before the execute phase |
898
-
899
- ### Description
900
-
901
- A fast-fix team ran in a live-session:
902
- ```
903
- 04:12:20 live-session.prompt_start 01_explore
904
- 04:12:51 live-session.prompt_done 01_explore (31s, completed)
905
- 04:12:51 live_agent.terminated 01_explore (status=cancelled)
906
- 04:12:51 task.completed 01_explore
907
- 04:12:51 run.cancelled: "This operation was aborted"
908
- ```
909
-
910
- Task `01_explore` completed successfully, but the run was cancelled before `02_execute` started.
911
-
912
- ### Possible causes
913
-
914
- 1. **Session concurrency limit** — only 1 active live-session, conflicting with parallel test operations
915
- 2. **User-initiated cancellation** — accidentally triggered
916
- 3. **Workflow phase transition bug** — does not trigger the next phase after explore completes
917
-
918
- ### Needs further investigation
919
-
920
- - Run the fast-fix team standalone (no concurrent operations)
921
- - Check live-session-runtime.ts for phase-transition logic
922
-
923
- ---
924
-
925
- ## Summary
926
-
927
- | # | Bug | Severity | Status | Category |
928
- |---|---|---|---|---|
929
- | 1 | Background workers timeout due to MiniMax 429 | 🔴 HIGH | ✅ Fixed — 429 now retries with fallback models via improved RETRYABLE_MODEL_FAILURE_PATTERNS | Code |
930
- | 2 | child-pi.ts does not detect 429, reports wrong "heartbeat dead" | 🔴 HIGH | ✅ Fixed — removed 429 fast-fail; let task-runner handle retry+fallback | Code |
931
- | 3 | background.log useless, does not capture worker output | 🟠 MEDIUM | ✅ Fixed — added PI_CREW_BACKGROUND_MODE flag + event logging to background.log | Observability |
932
- | 4 | worker-startup.ts missing rate_limited classification | 🟡 LOW | ✅ Fixed — added rate_limited + provider_error to StartupFailureClassification | Code |
933
- | 5 | Stale heartbeat notifications after prune | 🟡 LOW | ✅ Fixed — HeartbeatWatcher skips pruned runs via stateRoot existence check | UX |
934
- | 6 | Live-session foreground run cancelled when there are concurrent tool calls | 🟠 MEDIUM | ✅ Confirmed — concurrent calls interrupt live-session → outputLength:0 → caller_cancelled. Avoid concurrent team actions during foreground runs. | Runtime |
935
- | 7 | Async notifier "stale ctx" — dies, does not restart after Pi restart | 🔴 HIGH | ✅ Fixed — swallow stale error, isCurrent guard handles dormancy | Code |
936
- | 8 | Background child-process 300s timeout — child Pi hangs, zero output | 🟠 MEDIUM | ✅ Fixed — Root cause found (Bug #10): MINIMAX_API_KEY stripped by sanitizeEnvSecrets(). Allow-list in child-pi.ts preserves model provider API keys. Restart Pi to verify fix. | Code |
937
- | 9 | Executor hit yield limit — file write not completed | 🟡 LOW | 🔲 Open — executor hit 3 Yield Reminders and terminated before writing file. Task marked completed but artifact missing. | Runtime |
938
- | 10 | Child-process silent timeout — MINIMAX_API_KEY filtered out of child env | 🔴 HIGH | ✅ Fixed — sanitizeEnvSecrets() strips *API_KEY* vars. Allow-list in buildChildPiSpawnOptions preserves model provider keys (MINIMAX_*, OPENAI_*, etc.). See docs/fixes/bug-010-child-process-api-key-filtered.md | Code |
939
-
940
-
941
- | 11 | Background runner "spawn pi ENOENT" — pi binary not in PATH | 🔴 HIGH | ✅ Fixed — added resolvePiCliScript() call for non-Windows platforms in getPiSpawnCommand(). Restart Pi to verify. | Code |
942
- | 12 | Essential env vars (PATH) stripped - child Pi crashes with npm root -g error | HIGH | ✅ Fixed — added essential env vars (PATH, HOME, USER, etc.) to allow-list alongside model API keys. Restart Pi to verify. | Code |
943
- | 15 | Background runner receives SIGTERM ~3s after spawn from Pi infrastructure | 🟠 MEDIUM | ✅ Fixed — disabled async mode by default + ignore SIGTERM from Pi in background-runner | Runtime |
944
-
945
- ### Priority fix order
946
-
947
- 1. **Bug #1** — ✅ Fixed — 429 now retried with model fallback chain
948
- 2. **Bug #2** — ✅ Fixed — removed 429 fast-fail
949
- 3. **Bug #3** — ✅ Fixed — worker events now logged to background.log
950
- 4. **Bug #4** — ✅ Fixed — rate_limited + provider_error classification added
951
- 5. **Bug #5** — ✅ Fixed — HeartbeatWatcher skips pruned runs
952
- 6. **Bug #6** — ✅ Confirmed — concurrent tool calls cancel foreground runs; avoid concurrent team actions during runs
953
- 7. **Bug #7** — ✅ Fixed — async notifier handles stale ctx gracefully, isCurrent guard manages dormancy
954
- 8. **Bug #8/10** — ✅ Fixed — Bug #10 root cause: MINIMAX_API_KEY filtered out. Allow-list preserves model provider API keys for child processes.
955
- 9. **Bug #9** — ✅ Fixed — Added `needs_attention` task status. Workers that complete without calling `submit_result` now get `status: "needs_attention"` instead of `"completed"`, with ⚠ icon in UI.
956
- 10. **Bug #10** — ✅ Fixed — Added allow-list to sanitizeEnvSecrets in child-pi.ts to preserve model API keys (MINIMAX_*, OPENAI_*, etc.)
957
- 11. **Bug #11** — ✅ Fixed — resolvePiCliScript() added for non-Windows in getPiSpawnCommand() to fix ENOENT on spawn
958
- 12. **Bug #12** — ✅ Fixed — Essential env vars (PATH, HOME, USER, etc.) added to allow-list alongside model API keys
959
- 13. **Bug #13** — 🟠 MEDIUM — ✅ Fixed — Background runner dies after ~59s. 3-layer fix: (1) heartbeat mechanism prevents false repairs; (2) --max-old-space-size=512 limits V8 heap to prevent OOM; (3) SIGTERM/SIGINT handlers log async.failed event for diagnosis. Heartbeat includes memory stats (heapUsedMb, rssMb) for post-mortem.
960
- 14. **Bug #14** — 🔴 HIGH — ✅ Fixed — Infinite retry loop: needs_attention tasks had `queue: "blocked"` in task graph instead of `queue: "done"`, causing them to be re-scheduled indefinitely. Added `needs_attention` to the terminal status check in `withQueue()` in task-graph-scheduler.ts.
961
- 15. **Bug #15** — 🟠 MEDIUM — ✅ Fixed — Disabled async mode by default (runAsync=false). Background runners receive SIGTERM ~3s after spawn from Pi infrastructure because Node.js 22.22.0 setsid:true doesn't create a new session. Also added ignore-SIGTERM-from-Pi logic in background-runner.ts (A2 approach).