pi-crew 0.1.30 → 0.1.32

package/docs/architecture.md

@@ -1,164 +1,173 @@
-# pi-crew Architecture
-
-`pi-crew` is a Pi package for coordinated multi-agent work. It is intentionally durable-first: every run is represented on disk, every task has a state record, and child workers stream progress into JSONL/status files so foreground sessions, background jobs, dashboards, and later restarts all read the same source of truth.
-
-## Layers
-
-```text
-Pi extension layer
-  register tools, slash commands, widget/dashboard, notifier, lifecycle cleanup
-
-Runtime layer
-  team runner, task graph scheduler, child Pi process runner, async runner,
-  model fallback, policy engine, worktree manager, live-session experimental path
-
-State layer
-  .pi/teams/state/runs/{runId}/manifest.json
-  .pi/teams/state/runs/{runId}/tasks.json
-  .pi/teams/state/runs/{runId}/events.jsonl
-  .pi/teams/state/runs/{runId}/agents/{taskId}/status.json
-  .pi/teams/artifacts/{runId}/...
-```
-
-## Run flow
-
-```text
-user/team tool
-  │
-  ▼
-handleTeamTool(action=run)
-  ├─ discover agents/teams/workflows
-  ├─ validate team/workflow refs
-  ├─ create run manifest + task graph
-  ├─ write goal artifact
-  └─ choose foreground/session-bound or async/background mode
-        │
-        ├─ foreground: startForegroundRun() schedules executeTeamRun()
-        │
-        └─ async: spawnBackgroundTeamRun()
-              ├─ node --import jiti-register.mjs background-runner.ts
-              ├─ background-runner writes async.started + async.pid marker
-              └─ executeTeamRun()
-                    ├─ resolve ready task batch
-                    ├─ resolveBatchConcurrency() with hard cap
-                    ├─ runTeamTask() per task
-                    │    ├─ build prompt + dependency context
-                    │    ├─ choose configured Pi model candidates
-                    │    ├─ spawn child `pi` worker
-                    │    ├─ observe JSONL/stdout progress
-                    │    ├─ persist agent status/events/output
-                    │    └─ write result/log/transcript artifacts
-                    ├─ merge task updates monotonically
-                    ├─ write progress artifacts
-                    └─ synthesize policy closeout
-```
-
-## Extension layer
-
-`src/extension/register.ts` wires the package into Pi:
-
-- `team` tool and management actions.
-- Conflict-safe subagent tools: `crew_agent`, `crew_agent_result`, `crew_agent_steer`.
-- Claude-style aliases: `Agent`, `get_subagent_result`, `steer_subagent` when available.
-- Slash commands including `/team-run`, `/team-status`, `/team-dashboard`, `/team-doctor`, `/team-config`, `/team-summary`.
-- Active-only widget and optional dashboard/sidebar UI.
-- Foreground run scheduling and shutdown cleanup.
-- Async completion notifier and session-start active-run summary.
-
-The extension layer should remain thin: user input is normalized into tool parameters, then delegated to runtime/state modules.
-
-## Runtime layer
-
-### Team runner
-
-`src/runtime/team-runner.ts` drives workflow execution. It reads queued tasks, computes the ready set from the task graph, applies concurrency limits, runs a batch, then merges results back into the latest task state. Terminal task states are monotonic: stale parallel snapshots must not regress completed/failed/cancelled/skipped tasks back to queued/running.
-
-### Task runner
-
-`src/runtime/task-runner.ts` executes one task. It prepares workspace/worktree context, renders a task prompt, chooses model candidates from Pi configuration, launches a child Pi process by default, and writes result artifacts. Scaffold mode is explicit dry-run only.
-
-### Child Pi runtime
-
-`src/runtime/child-pi.ts` is the default worker runtime. It:
-
-- launches real `pi` child processes,
-- hides Windows console windows with `windowsHide: true`,
-- streams JSONL output into transcripts,
-- compacts noisy message updates,
-- isolates observer callback failures so progress persistence cannot kill orchestration,
-- applies post-exit stdio guards for late output.
-
-### Async background runner
-
-`src/runtime/async-runner.ts` spawns detached background runs. Installed packages use an absolute `jiti-register.mjs` loader path because Node strip-types refuses TypeScript under `node_modules`. The runner fail-fasts if jiti is missing, and writes `async.pid` once startup begins so the parent can distinguish a healthy start from an early import crash.
-
-### Concurrency and policy
-
-`src/runtime/concurrency.ts` picks batch size from explicit limits, team settings, workflow settings, or built-in defaults. User-provided `limits.maxConcurrentWorkers` is hard-capped by default to prevent local DoS; `limits.allowUnboundedConcurrency=true` is an explicit opt-out and emits an observability event.
-
-`src/runtime/policy-engine.ts` applies closeout and safety policy decisions such as limit exceeded, failed task blocking, stale workers, and green-contract failures.
-
-### Model routing
-
-Model choice is based on Pi's current configuration/model registry, not hardcoded providers. Task and agent records persist model attempts and routing metadata so dashboards/status can show requested model, selected model, fallback chain, and fallback reason.
-
-## State layer
-
-Run state is under:
-
-```text
-.pi/teams/state/runs/{runId}/
-  manifest.json        run metadata/status/artifacts/async pid
-  tasks.json           task graph and per-task status
-  events.jsonl         append-only run events
-  events.jsonl.seq     event sequence cache
-  agents.json          aggregate agent cache
-  async.pid            background startup marker
-  agents/{taskId}/
-    status.json        per-agent status source
-    events.jsonl       per-agent event stream
-    output.log         compact worker output
-    sidechain.output.jsonl
-    live-control.jsonl
-```
-
-Artifacts are under:
-
-```text
-.pi/teams/artifacts/{runId}/
-  goal.md
-  prompts/{taskId}.md
-  results/{taskId}.txt
-  logs/{taskId}.log
-  transcripts/{taskId}.jsonl
-  metadata/*.json
-  progress.md
-  summary.md
-```
-
-Atomic writes use temp-file replace with retry for transient Windows `EPERM`/`EBUSY`/`EACCES`. JSONL append paths are best-effort where used for observers/progress; write failures must not crash child output parsing.
-
-## UI and observability
-
-- The persistent widget shows active runs only.
-- Stale async runs with dead background pids are hidden from the active widget.
-- `/team-status` is the canonical detailed state view and can mark stale active async runs failed.
-- `/team-dashboard` provides history and details.
-- Powerbar publishing is optional and event-compatible.
-- Transcript viewer is file-backed so it works for foreground and async runs.
-
-## Lifecycle and cleanup
-
-Foreground runs are session-bound and should be interrupted on session shutdown or session switch. Only explicit `async: true` runs are allowed to survive the Pi session. Runtime cleanup is registered through Pi lifecycle hooks and a global reload cleanup guard.
-
-## Configuration
-
-Key config sections:
-
-- `runtime`: `auto`, `child-process`, `scaffold`, experimental `live-session`.
-- `limits`: concurrency/task/depth safety controls.
-- `ui`: widget/dashboard/powerbar/model-token display settings.
-- `agents`: builtin overrides for models/fallbacks/tools.
-- `autonomous`: policy injection/profile for proactive team delegation.
-
-See `usage.md`, `resource-formats.md`, `runtime-flow.md`, and `live-mailbox-runtime.md` for operational details.
+# pi-crew Architecture
+
+`pi-crew` is a Pi package for coordinated multi-agent work. It is intentionally durable-first: every run is represented on disk, every task has a state record, and child workers stream progress into JSONL/status files so foreground sessions, background jobs, dashboards, and later restarts all read the same source of truth.
+
+## Layers
+
+```text
+Pi extension layer
+  register tools, slash commands, widget/dashboard, notifier, lifecycle cleanup
+
+Runtime layer
+  team runner, task graph scheduler, child Pi process runner, async runner,
+  model fallback, policy engine, worktree manager, live-session experimental path
+
+State layer (project root resolves to <crewRoot>:
+  - .crew/             when no .pi/ exists in the repo (default)
+  - .pi/teams/         when the repo already has .pi/ (legacy reuse))
+  <crewRoot>/state/runs/{runId}/manifest.json
+  <crewRoot>/state/runs/{runId}/tasks.json
+  <crewRoot>/state/runs/{runId}/events.jsonl
+  <crewRoot>/state/runs/{runId}/agents/{taskId}/status.json
+  <crewRoot>/artifacts/{runId}/...
+```
+
+## Run flow
+
+```text
+user/team tool
+  │
+  ▼
+handleTeamTool(action=run)
+  ├─ discover agents/teams/workflows
+  ├─ validate team/workflow refs
+  ├─ create run manifest + task graph
+  ├─ write goal artifact
+  └─ choose foreground/session-bound or async/background mode
+        │
+        ├─ foreground: startForegroundRun() schedules executeTeamRun()
+        │
+        └─ async: spawnBackgroundTeamRun()
+              ├─ node --import jiti-register.mjs background-runner.ts
+              ├─ background-runner writes async.started + async.pid marker
+              └─ executeTeamRun()
+                    ├─ resolve ready task batch
+                    ├─ resolveBatchConcurrency() with hard cap
+                    ├─ runTeamTask() per task
+                    │    ├─ build prompt + dependency context
+                    │    ├─ choose configured Pi model candidates
+                    │    ├─ spawn child `pi` worker
+                    │    ├─ observe JSONL/stdout progress
+                    │    ├─ persist agent status/events/output
+                    │    └─ write result/log/transcript artifacts
+                    ├─ merge task updates monotonically
+                    ├─ write progress artifacts
+                    └─ synthesize policy closeout
+```
+
+## Extension layer
+
+`src/extension/register.ts` wires the package into Pi:
+
+- `team` tool and management actions.
+- Conflict-safe subagent tools: `crew_agent`, `crew_agent_result`, `crew_agent_steer`.
+- Claude-style aliases: `Agent`, `get_subagent_result`, `steer_subagent` when available.
+- Slash commands including `/team-run`, `/team-status`, `/team-dashboard`, `/team-doctor`, `/team-config`, `/team-summary`.
+- Active-only widget and optional dashboard/sidebar UI.
+- Foreground run scheduling and shutdown cleanup.
+- Async completion notifier and session-start active-run summary.
+
+The extension layer should remain thin: user input is normalized into tool parameters, then delegated to runtime/state modules.
+
+## Runtime layer
+
+### Team runner
+
+`src/runtime/team-runner.ts` drives workflow execution. It reads queued tasks, computes the ready set from the task graph, applies concurrency limits, runs a batch, then merges results back into the latest task state. Terminal task states are monotonic: stale parallel snapshots must not regress completed/failed/cancelled/skipped tasks back to queued/running.
+
+### Task runner
+
+`src/runtime/task-runner.ts` executes one task. It prepares workspace/worktree context, renders a task prompt, chooses model candidates from Pi configuration, launches a child Pi process by default, and writes result artifacts. Scaffold mode is explicit dry-run only.
+
+### Child Pi runtime
+
+`src/runtime/child-pi.ts` is the default worker runtime. It:
+
+- launches real `pi` child processes,
+- hides Windows console windows with `windowsHide: true`,
+- streams JSONL output into transcripts,
+- compacts noisy message updates,
+- isolates observer callback failures so progress persistence cannot kill orchestration,
+- applies post-exit stdio guards for late output.
+
+### Async background runner
+
+`src/runtime/async-runner.ts` spawns detached background runs. Installed packages use an absolute `jiti-register.mjs` loader path because Node strip-types refuses TypeScript under `node_modules`. The runner fail-fasts if jiti is missing, and writes `async.pid` once startup begins so the parent can distinguish a healthy start from an early import crash.
+
+### Concurrency and policy
+
+`src/runtime/concurrency.ts` picks batch size from explicit limits, team settings, workflow settings, or built-in defaults. User-provided `limits.maxConcurrentWorkers` is hard-capped by default to prevent local DoS; `limits.allowUnboundedConcurrency=true` is an explicit opt-out and emits an observability event.
+
+`src/runtime/policy-engine.ts` applies closeout and safety policy decisions such as limit exceeded, failed task blocking, stale workers, and green-contract failures.
+
+### Model routing
+
+Model choice is based on Pi's current configuration/model registry, not hardcoded providers. Task and agent records persist model attempts and routing metadata so dashboards/status can show requested model, selected model, fallback chain, and fallback reason.
+
+## State layer
+
+Run state is under `<crewRoot>` (`.crew/` for new projects, or `.pi/teams/` when the repo already has `.pi/`):
+
+```text
+<crewRoot>/state/runs/{runId}/
+  manifest.json        run metadata/status/artifacts/async pid
+  tasks.json           task graph and per-task status
+  events.jsonl         append-only run events
+  events.jsonl.seq     event sequence cache
+  agents.json          aggregate agent cache
+  async.pid            background startup marker
+  agents/{taskId}/
+    status.json        per-agent status source
+    events.jsonl       per-agent event stream
+    output.log         compact worker output
+    sidechain.output.jsonl
+    live-control.jsonl
+```
+
+Artifacts are under:
+
+```text
+<crewRoot>/artifacts/{runId}/
+  goal.md
+  prompts/{taskId}.md
+  results/{taskId}.txt
+  logs/{taskId}.log
+  transcripts/{taskId}.jsonl
+  metadata/*.json
+  progress.md
+  summary.md
+```
+
+`<crewRoot>` resolution is centralised in `src/utils/paths.ts#projectCrewRoot()`:
+
+- if `<repoRoot>/.pi/` already exists, return `<repoRoot>/.pi/teams/` (legacy reuse, no parallel `.crew/`)
+- otherwise return `<repoRoot>/.crew/` (default for fresh projects)
+
+User-global fallback (when no project root is detected) lives under `~/.pi/agent/extensions/pi-crew/`.
+
+Atomic writes use temp-file replace with retry for transient Windows `EPERM`/`EBUSY`/`EACCES`. JSONL append paths are best-effort where used for observers/progress; write failures must not crash child output parsing.
+
+## UI and observability
+
+- The persistent widget shows active runs only.
+- Stale async runs with dead background pids are hidden from the active widget.
+- `/team-status` is the canonical detailed state view and can mark stale active async runs failed.
+- `/team-dashboard` provides history and details.
+- Powerbar publishing is optional and event-compatible.
+- Transcript viewer is file-backed so it works for foreground and async runs.
+
+## Lifecycle and cleanup
+
+Foreground runs are session-bound and should be interrupted on session shutdown or session switch. Only explicit `async: true` runs are allowed to survive the Pi session. Runtime cleanup is registered through Pi lifecycle hooks and a global reload cleanup guard.
+
+## Configuration
+
+Key config sections:
+
+- `runtime`: `auto`, `child-process`, `scaffold`, experimental `live-session`.
+- `limits`: concurrency/task/depth safety controls.
+- `ui`: widget/dashboard/powerbar/model-token display settings.
+- `agents`: builtin overrides for models/fallbacks/tools.
+- `autonomous`: policy injection/profile for proactive team delegation.
+
+See `usage.md`, `resource-formats.md`, `runtime-flow.md`, and `live-mailbox-runtime.md` for operational details.