pi-crew 0.1.37 → 0.1.39

package/docs/research-source-pi-crew-reference.md

@@ -1,174 +1,174 @@
-# Research: `source/pi-crew` as New Reference Source
-
-Date: 2026-04-29
-Reference source: `D:/my/my_project/source/pi-crew` (`@melihmucuk/pi-crew@1.0.14`, commit `c0631a3`)
-Current target: `D:/my/my_project/pi-crew` (`pi-crew@0.1.34`)
-Research run: `team_20260429091311_8047706b`
-
-> Note: the parallel research run produced useful artifacts, but child workers were marked failed because they did not exit within 5s after their final assistant message. The source audit content was still captured in result/shared artifacts.
-
-## Executive Summary
-
-`source/pi-crew` is a compact, in-process subagent orchestration extension. It is not a team/workflow engine; instead, it focuses on fast non-blocking subagent sessions, owner-routed steering-message delivery, interactive subagents, and context-overflow recovery. It is valuable as a reference for **session-native subagent runtime**, **delivery semantics**, and **minimal interactive worker UX**.
-
-Current `pi-crew` is more powerful and durable: child Pi workers, teams/workflows, task graph scheduling, worktrees, mailbox, event logs, dashboard, notifications, and recovery state. The best path is not replacement; it is selective porting of patterns into `pi-crew`'s existing `live-session-runtime` / `SubagentManager` as an optional session-native lane.
-
-## Source File Map
-
-| Area | Reference files |
-|---|---|
-| Extension entry/session hooks | `source/pi-crew/extension/index.ts` |
-| Runtime singleton | `source/pi-crew/extension/runtime/crew-runtime.ts` |
-| Delivery routing | `source/pi-crew/extension/runtime/delivery-coordinator.ts` |
-| State model/registry | `source/pi-crew/extension/runtime/subagent-state.ts`, `source/pi-crew/extension/runtime/subagent-registry.ts` |
-| Overflow recovery | `source/pi-crew/extension/runtime/overflow-recovery.ts` |
-| Session bootstrap | `source/pi-crew/extension/bootstrap-session.ts` |
-| Agent discovery | `source/pi-crew/extension/agent-discovery.ts` |
-| Tool registration | `source/pi-crew/extension/integration/register-tools.ts`, `source/pi-crew/extension/integration/tools/*.ts` |
-| Message renderers | `source/pi-crew/extension/integration/register-renderers.ts` |
-| Message formatting | `source/pi-crew/extension/subagent-messages.ts` |
-| Status widget | `source/pi-crew/extension/status-widget.ts` |
-| Architecture doc | `source/pi-crew/docs/architecture.md` |
-
-## Architecture Observations
-
-### Reference `source/pi-crew`
-
-- Process-level singleton `CrewRuntime` survives Pi runtime/session replacement and rebinds on `session_start`.
-- Subagents are in-process SDK `AgentSession`s created with `createAgentSession()`.
-- Parent/child linkage uses `SessionManager.newSession({ parentSession })`.
-- Subagent resource loading filters out the pi-crew extension through `extensionsOverride` to prevent recursive `crew_spawn` loops.
-- Results are delivered through Pi-native `sendMessage()` with explicit idle/streaming semantics.
-- Interactive subagents are first-class: `interactive: true` workers enter `waiting`; parent continues with `crew_respond`; cleanup is explicit with `crew_done`.
-- Overflow recovery tracks `agent_end`, `compaction_start/end`, and `auto_retry_start/end` events around `session.prompt()`.
-- State is in-memory only; subagent session files remain for post-hoc `/resume` inspection.
-
-### Current `pi-crew`
-
-- Primary runtime is child Pi process execution with durable `.crew/state` manifests and artifacts.
-- It has workflow/team abstractions, task graphs, worktree support, event log, mailbox, dashboard panes, render scheduler, notifications, and diagnostic exports.
-- It already has `live-session-runtime.ts`, but the current product surface centers on durable child-process workers rather than interactive in-process subagents.
-
-## Extension API Patterns Worth Reusing
-
-| Pattern | Reference source | Why it matters for current `pi-crew` |
-|---|---|---|
-| Owner-routed delivery by `sessionManager.getSessionId()` | `delivery-coordinator.ts` | Avoids sending async worker results to the wrong active session after `/resume`, `/new`, `/fork`, or multi-session use. |
-| Idle vs streaming delivery split | `subagent-messages.ts`, `delivery-coordinator.ts` | Prevents messages from getting stuck: idle sessions need `triggerTurn`; streaming sessions need `deliverAs: "steer"`. |
-| Deferred pending flush via `setTimeout(0)` | `delivery-coordinator.ts` | Avoids lost JSONL/custom-message persistence during resume before listeners reconnect. |
-| `extensionsOverride` filter | `bootstrap-session.ts` | Required for any in-process worker lane to prevent recursive subagent spawning. |
-| Fire-and-forget interactive response | `crew-respond.ts`, `crew-runtime.ts` | Lets parent stay responsive while an interactive worker continues in background. |
-| No duplicate done message | `crew-done.ts` | Avoids repeating the last subagent response during cleanup. |
-| Source-specific abort reasons | `crew-abort.ts`, `index.ts` shutdown handlers | Better diagnostics than generic "aborted by user". |
-| Emergency unrestricted abort command | `register-command.ts` | Useful escape hatch distinct from owner-scoped tool actions. |
-| Overflow tracker around SDK prompt | `overflow-recovery.ts` | Better UX for context overflow/compaction/retry in session-native workers. |
-
-## Key Differences / Non-Goals
-
-| Dimension | Reference `source/pi-crew` | Current `pi-crew` |
-|---|---|---|
-| Runtime | In-process `AgentSession` | Child Pi processes + durable orchestration |
-| State | In-memory map | Durable manifests/event logs/artifacts |
-| Scope | Flat subagent spawn/respond/done | Teams, workflows, task graph, worktrees |
-| Result UX | Pi steering/custom messages | Tool results, mailbox, dashboard, async status |
-| Interactive workers | Native | Not yet first-class |
-| Worktree isolation | None | First-class |
-| Replay/restart | Limited | Strong durable recovery |
-
-Do **not** replace the current runtime wholesale. Reference `source/pi-crew` lacks durable state, worktrees, workflow scheduling, artifact indexing, and the Phase 8 operator experience. Its best value is a narrower session-native execution lane and delivery correctness patterns.
-
-## Recommendations
-
-### P0 — Adopt Delivery Semantics for Async/Live Results
-
-Implement or adapt a small owner-routed delivery coordinator in current `pi-crew`:
-
-- Key by owner `sessionId`, not session file.
-- Queue pending messages when owner inactive.
-- On `session_start`, flush pending messages on next macrotask.
-- Use idle/streaming split:
-  - idle: `sendMessage(payload, { triggerTurn: true })`
-  - streaming: `sendMessage(payload, { deliverAs: "steer", triggerTurn: true })`
-- Keep current mailbox/event-log as durable source of truth; use delivery coordinator only for live UX.
-
-Likely target files:
-
-- `pi-crew/src/extension/register.ts`
-- `pi-crew/src/runtime/subagent-manager.ts`
-- `pi-crew/src/runtime/live-session-runtime.ts`
-- `pi-crew/src/extension/notification-router.ts`
-
-### P1 — Add Optional Session-Native Subagent Lane
-
-Build an opt-in lane on top of existing `live-session-runtime.ts` rather than changing the default child-process runtime:
-
-- `runtime.mode = "child-process" | "live-session" | "auto"` already exists conceptually; tighten semantics.
-- Use `SessionManager.newSession({ parentSession })` and `createAgentSession()` for in-process workers.
-- Filter `pi-crew` out of subagent resource loader extensions.
-- Persist minimal metadata to existing `.crew/state` so dashboards/recovery still work.
-
-This can reduce process startup overhead and blank console issues, while preserving child-process isolation as the safe default.
-
-### P1 — Introduce Interactive Worker Semantics
-
-Add first-class interactive subagents without disrupting teams:
-
-- New status: `waiting` for interactive background workers.
-- `crew_agent_respond` / `crew_agent_done` or extend existing `crew_agent_steer` semantics.
-- Fire-and-forget response: parent tool returns immediately; worker response arrives as mailbox/steering message.
-- `done` performs cleanup only; no duplicate response.
-
-Likely target files:
-
-- `pi-crew/src/runtime/crew-agent-records.ts`
-- `pi-crew/src/runtime/subagent-manager.ts`
-- `pi-crew/src/extension/registration/subagent-tools.ts`
-- `pi-crew/src/state/mailbox.ts`
-- `pi-crew/src/ui/dashboard-panes/agents-pane.ts`
-
-### P2 — Port Overflow Recovery Tracker for Live Sessions
-
-For session-native workers, wrap `AgentSession.prompt()` with an event tracker similar to `source/pi-crew/extension/runtime/overflow-recovery.ts`:
-
-- Track `compaction_start/end` and `auto_retry_start/end`.
-- Report recovered context overflow separately from hard failure.
-- Emit durable event-log records and dashboard health hints.
-
-This should not apply to child Pi workers directly; they already have process/transcript supervision.
-
-### P2 — Improve Abort Reason Taxonomy
-
-Adopt explicit abort source reasons across all worker paths:
-
-- tool-triggered abort
-- command-triggered emergency abort
-- session quit cleanup
-- session replacement detach/deactivate
-- watchdog timeout
-- stale heartbeat kill
-
-This improves diagnostics, notification routing, and Phase 9 reliability work.
-
-## Risks
-
-- In-process sessions reduce OS/process isolation; failures or leaks may affect the parent Pi process.
-- `extensionsOverride` is mandatory; missing it risks recursive subagent spawning.
-- Pi SDK internals may shift; keep this lane optional and covered by integration tests.
-- Delivery semantics must not bypass durable mailbox/event log; live messages are convenience, not source of truth.
-- Interactive workers can linger in memory; require TTL/status visibility and explicit cleanup.
-
-## Suggested Follow-Up Plan
-
-1. Write a focused design doc: `docs/research-session-native-runtime-plan.md`.
-2. Spike delivery coordinator only; no runtime swap.
-3. Add tests for idle/streaming/inactive owner delivery behavior.
-4. Add optional `live-session` worker lane behind config.
-5. Add interactive worker status/actions after live delivery is stable.
-
-## Research Artifacts
-
-- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/01_discover.txt`
-- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/02_explore-shard-1.txt`
-- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/03_explore-shard-2.txt`
-- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/04_explore-shard-3.txt`
-- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/batches/01_discover+02_explore-shard-1+03_explore-shard-2+04_explore-shard-3.md`
+# Research: `source/pi-crew` as New Reference Source
+
+Date: 2026-04-29
+Reference source: `D:/my/my_project/source/pi-crew` (`@melihmucuk/pi-crew@1.0.14`, commit `c0631a3`)
+Current target: `D:/my/my_project/pi-crew` (`pi-crew@0.1.34`)
+Research run: `team_20260429091311_8047706b`
+
+> Note: the parallel research run produced useful artifacts, but child workers were marked failed because they did not exit within 5s after their final assistant message. The source audit content was still captured in result/shared artifacts.
+
+## Executive Summary
+
+`source/pi-crew` is a compact, in-process subagent orchestration extension. It is not a team/workflow engine; instead, it focuses on fast non-blocking subagent sessions, owner-routed steering-message delivery, interactive subagents, and context-overflow recovery. It is valuable as a reference for **session-native subagent runtime**, **delivery semantics**, and **minimal interactive worker UX**.
+
+Current `pi-crew` is more powerful and durable: child Pi workers, teams/workflows, task graph scheduling, worktrees, mailbox, event logs, dashboard, notifications, and recovery state. The best path is not replacement; it is selective porting of patterns into `pi-crew`'s existing `live-session-runtime` / `SubagentManager` as an optional session-native lane.
+
+## Source File Map
+
+| Area | Reference files |
+|---|---|
+| Extension entry/session hooks | `source/pi-crew/extension/index.ts` |
+| Runtime singleton | `source/pi-crew/extension/runtime/crew-runtime.ts` |
+| Delivery routing | `source/pi-crew/extension/runtime/delivery-coordinator.ts` |
+| State model/registry | `source/pi-crew/extension/runtime/subagent-state.ts`, `source/pi-crew/extension/runtime/subagent-registry.ts` |
+| Overflow recovery | `source/pi-crew/extension/runtime/overflow-recovery.ts` |
+| Session bootstrap | `source/pi-crew/extension/bootstrap-session.ts` |
+| Agent discovery | `source/pi-crew/extension/agent-discovery.ts` |
+| Tool registration | `source/pi-crew/extension/integration/register-tools.ts`, `source/pi-crew/extension/integration/tools/*.ts` |
+| Message renderers | `source/pi-crew/extension/integration/register-renderers.ts` |
+| Message formatting | `source/pi-crew/extension/subagent-messages.ts` |
+| Status widget | `source/pi-crew/extension/status-widget.ts` |
+| Architecture doc | `source/pi-crew/docs/architecture.md` |
+
+## Architecture Observations
+
+### Reference `source/pi-crew`
+
+- Process-level singleton `CrewRuntime` survives Pi runtime/session replacement and rebinds on `session_start`.
+- Subagents are in-process SDK `AgentSession`s created with `createAgentSession()`.
+- Parent/child linkage uses `SessionManager.newSession({ parentSession })`.
+- Subagent resource loading filters out the pi-crew extension through `extensionsOverride` to prevent recursive `crew_spawn` loops.
+- Results are delivered through Pi-native `sendMessage()` with explicit idle/streaming semantics.
+- Interactive subagents are first-class: `interactive: true` workers enter `waiting`; parent continues with `crew_respond`; cleanup is explicit with `crew_done`.
+- Overflow recovery tracks `agent_end`, `compaction_start/end`, and `auto_retry_start/end` events around `session.prompt()`.
+- State is in-memory only; subagent session files remain for post-hoc `/resume` inspection.
+
+### Current `pi-crew`
+
+- Primary runtime is child Pi process execution with durable `.crew/state` manifests and artifacts.
+- It has workflow/team abstractions, task graphs, worktree support, event log, mailbox, dashboard panes, render scheduler, notifications, and diagnostic exports.
+- It already has `live-session-runtime.ts`, but the current product surface centers on durable child-process workers rather than interactive in-process subagents.
+
+## Extension API Patterns Worth Reusing
+
+| Pattern | Reference source | Why it matters for current `pi-crew` |
+|---|---|---|
+| Owner-routed delivery by `sessionManager.getSessionId()` | `delivery-coordinator.ts` | Avoids sending async worker results to the wrong active session after `/resume`, `/new`, `/fork`, or multi-session use. |
+| Idle vs streaming delivery split | `subagent-messages.ts`, `delivery-coordinator.ts` | Prevents messages from getting stuck: idle sessions need `triggerTurn`; streaming sessions need `deliverAs: "steer"`. |
+| Deferred pending flush via `setTimeout(0)` | `delivery-coordinator.ts` | Avoids lost JSONL/custom-message persistence during resume before listeners reconnect. |
+| `extensionsOverride` filter | `bootstrap-session.ts` | Required for any in-process worker lane to prevent recursive subagent spawning. |
+| Fire-and-forget interactive response | `crew-respond.ts`, `crew-runtime.ts` | Lets parent stay responsive while an interactive worker continues in background. |
+| No duplicate done message | `crew-done.ts` | Avoids repeating the last subagent response during cleanup. |
+| Source-specific abort reasons | `crew-abort.ts`, `index.ts` shutdown handlers | Better diagnostics than generic "aborted by user". |
+| Emergency unrestricted abort command | `register-command.ts` | Useful escape hatch distinct from owner-scoped tool actions. |
+| Overflow tracker around SDK prompt | `overflow-recovery.ts` | Better UX for context overflow/compaction/retry in session-native workers. |
+
+## Key Differences / Non-Goals
+
+| Dimension | Reference `source/pi-crew` | Current `pi-crew` |
+|---|---|---|
+| Runtime | In-process `AgentSession` | Child Pi processes + durable orchestration |
+| State | In-memory map | Durable manifests/event logs/artifacts |
+| Scope | Flat subagent spawn/respond/done | Teams, workflows, task graph, worktrees |
+| Result UX | Pi steering/custom messages | Tool results, mailbox, dashboard, async status |
+| Interactive workers | Native | Not yet first-class |
+| Worktree isolation | None | First-class |
+| Replay/restart | Limited | Strong durable recovery |
+
+Do **not** replace the current runtime wholesale. Reference `source/pi-crew` lacks durable state, worktrees, workflow scheduling, artifact indexing, and the Phase 8 operator experience. Its best value is a narrower session-native execution lane and delivery correctness patterns.
+
+## Recommendations
+
+### P0 — Adopt Delivery Semantics for Async/Live Results
+
+Implement or adapt a small owner-routed delivery coordinator in current `pi-crew`:
+
+- Key by owner `sessionId`, not session file.
+- Queue pending messages when owner inactive.
+- On `session_start`, flush pending messages on next macrotask.
+- Use idle/streaming split:
+  - idle: `sendMessage(payload, { triggerTurn: true })`
+  - streaming: `sendMessage(payload, { deliverAs: "steer", triggerTurn: true })`
+- Keep current mailbox/event-log as durable source of truth; use delivery coordinator only for live UX.
+
+Likely target files:
+
+- `pi-crew/src/extension/register.ts`
+- `pi-crew/src/runtime/subagent-manager.ts`
+- `pi-crew/src/runtime/live-session-runtime.ts`
+- `pi-crew/src/extension/notification-router.ts`
+
+### P1 — Add Optional Session-Native Subagent Lane
+
+Build an opt-in lane on top of existing `live-session-runtime.ts` rather than changing the default child-process runtime:
+
+- `runtime.mode = "child-process" | "live-session" | "auto"` already exists conceptually; tighten semantics.
+- Use `SessionManager.newSession({ parentSession })` and `createAgentSession()` for in-process workers.
+- Filter `pi-crew` out of subagent resource loader extensions.
+- Persist minimal metadata to existing `.crew/state` so dashboards/recovery still work.
+
+This can reduce process startup overhead and blank console issues, while preserving child-process isolation as the safe default.
+
+### P1 — Introduce Interactive Worker Semantics
+
+Add first-class interactive subagents without disrupting teams:
+
+- New status: `waiting` for interactive background workers.
+- `crew_agent_respond` / `crew_agent_done` or extend existing `crew_agent_steer` semantics.
+- Fire-and-forget response: parent tool returns immediately; worker response arrives as mailbox/steering message.
+- `done` performs cleanup only; no duplicate response.
+
+Likely target files:
+
+- `pi-crew/src/runtime/crew-agent-records.ts`
+- `pi-crew/src/runtime/subagent-manager.ts`
+- `pi-crew/src/extension/registration/subagent-tools.ts`
+- `pi-crew/src/state/mailbox.ts`
+- `pi-crew/src/ui/dashboard-panes/agents-pane.ts`
+
+### P2 — Port Overflow Recovery Tracker for Live Sessions
+
+For session-native workers, wrap `AgentSession.prompt()` with an event tracker similar to `source/pi-crew/extension/runtime/overflow-recovery.ts`:
+
+- Track `compaction_start/end` and `auto_retry_start/end`.
+- Report recovered context overflow separately from hard failure.
+- Emit durable event-log records and dashboard health hints.
+
+This should not apply to child Pi workers directly; they already have process/transcript supervision.
+
+### P2 — Improve Abort Reason Taxonomy
+
+Adopt explicit abort source reasons across all worker paths:
+
+- tool-triggered abort
+- command-triggered emergency abort
+- session quit cleanup
+- session replacement detach/deactivate
+- watchdog timeout
+- stale heartbeat kill
+
+This improves diagnostics, notification routing, and Phase 9 reliability work.
+
+## Risks
+
+- In-process sessions reduce OS/process isolation; failures or leaks may affect the parent Pi process.
+- `extensionsOverride` is mandatory; missing it risks recursive subagent spawning.
+- Pi SDK internals may shift; keep this lane optional and covered by integration tests.
+- Delivery semantics must not bypass durable mailbox/event log; live messages are convenience, not source of truth.
+- Interactive workers can linger in memory; require TTL/status visibility and explicit cleanup.
+
+## Suggested Follow-Up Plan
+
+1. Write a focused design doc: `docs/research-session-native-runtime-plan.md`.
+2. Spike delivery coordinator only; no runtime swap.
+3. Add tests for idle/streaming/inactive owner delivery behavior.
+4. Add optional `live-session` worker lane behind config.
+5. Add interactive worker status/actions after live delivery is stable.
+
+## Research Artifacts
+
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/01_discover.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/02_explore-shard-1.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/03_explore-shard-2.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/04_explore-shard-3.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/batches/01_discover+02_explore-shard-1+03_explore-shard-2+04_explore-shard-3.md`

package/docs/resource-formats.md

@@ -5,9 +5,10 @@
 Location:
 
 ```text
-agents/{name}.md
-~/.pi/agent/agents/{name}.md
-.pi/agents/{name}.md
+agents/{name}.md                                # builtin (in this package)
+~/.pi/agent/agents/{name}.md                    # user-global
+.crew/agents/{name}.md                          # project (new layout)
+.pi/teams/agents/{name}.md                      # project (legacy layout when .pi/ exists)
 ```
 
 Format:
@@ -80,7 +81,7 @@ category: implementation
 Role line:
 
 ```text
-- {role-name}: agent={agent-name} optional description
+- {role-name}: agent={agent-name} [model={provider/model}] [skills={a,b}|false] [maxConcurrency={n}] optional description
 ```
 
 ## Workflow files
@@ -88,9 +89,10 @@ Role line:
 Location:
 
 ```text
-workflows/{name}.workflow.md
-~/.pi/agent/workflows/{name}.workflow.md
-.pi/workflows/{name}.workflow.md
+workflows/{name}.workflow.md                    # builtin (in this package)
+~/.pi/agent/workflows/{name}.workflow.md        # user-global
+.crew/workflows/{name}.workflow.md              # project (new layout)
+.pi/teams/workflows/{name}.workflow.md          # project (legacy layout when .pi/ exists)
 ```
 
 Format:
@@ -129,4 +131,4 @@ Step fields:
 | `worktree` | `true`/`false` metadata |
 | `verify` | `true`/`false` verification marker |
 
-Avoid using level-2 headings (`##`) inside task bodies because they delimit workflow steps.
+Each step starts with `## step-id` followed by recognized step metadata such as `role:` before the blank line. Level-2 headings inside task bodies are preserved unless they look like a step section with recognized metadata; use `###` or lower for maximum compatibility.