pi-crew 0.1.34 → 0.1.35

package/docs/research-source-pi-crew-reference.md

@@ -0,0 +1,174 @@
+# Research: `source/pi-crew` as New Reference Source
+
+Date: 2026-04-29
+Reference source: `D:/my/my_project/source/pi-crew` (`@melihmucuk/pi-crew@1.0.14`, commit `c0631a3`)
+Current target: `D:/my/my_project/pi-crew` (`pi-crew@0.1.34`)
+Research run: `team_20260429091311_8047706b`
+
+> Note: the parallel research run produced useful artifacts, but child workers were marked failed because they did not exit within 5s after their final assistant message. The source audit content was still captured in result/shared artifacts.
+
+## Executive Summary
+
+`source/pi-crew` is a compact, in-process subagent orchestration extension. It is not a team/workflow engine; instead, it focuses on fast non-blocking subagent sessions, owner-routed steering-message delivery, interactive subagents, and context-overflow recovery. It is valuable as a reference for **session-native subagent runtime**, **delivery semantics**, and **minimal interactive worker UX**.
+
+Current `pi-crew` is more powerful and durable: child Pi workers, teams/workflows, task graph scheduling, worktrees, mailbox, event logs, dashboard, notifications, and recovery state. The best path is not replacement; it is selective porting of patterns into `pi-crew`'s existing `live-session-runtime` / `SubagentManager` as an optional session-native lane.
+
+## Source File Map
+
+| Area | Reference files |
+|---|---|
+| Extension entry/session hooks | `source/pi-crew/extension/index.ts` |
+| Runtime singleton | `source/pi-crew/extension/runtime/crew-runtime.ts` |
+| Delivery routing | `source/pi-crew/extension/runtime/delivery-coordinator.ts` |
+| State model/registry | `source/pi-crew/extension/runtime/subagent-state.ts`, `source/pi-crew/extension/runtime/subagent-registry.ts` |
+| Overflow recovery | `source/pi-crew/extension/runtime/overflow-recovery.ts` |
+| Session bootstrap | `source/pi-crew/extension/bootstrap-session.ts` |
+| Agent discovery | `source/pi-crew/extension/agent-discovery.ts` |
+| Tool registration | `source/pi-crew/extension/integration/register-tools.ts`, `source/pi-crew/extension/integration/tools/*.ts` |
+| Message renderers | `source/pi-crew/extension/integration/register-renderers.ts` |
+| Message formatting | `source/pi-crew/extension/subagent-messages.ts` |
+| Status widget | `source/pi-crew/extension/status-widget.ts` |
+| Architecture doc | `source/pi-crew/docs/architecture.md` |
+
+## Architecture Observations
+
+### Reference `source/pi-crew`
+
+- Process-level singleton `CrewRuntime` survives Pi runtime/session replacement and rebinds on `session_start`.
+- Subagents are in-process SDK `AgentSession`s created with `createAgentSession()`.
+- Parent/child linkage uses `SessionManager.newSession({ parentSession })`.
+- Subagent resource loading filters out the pi-crew extension through `extensionsOverride` to prevent recursive `crew_spawn` loops.
+- Results are delivered through Pi-native `sendMessage()` with explicit idle/streaming semantics.
+- Interactive subagents are first-class: `interactive: true` workers enter `waiting`; parent continues with `crew_respond`; cleanup is explicit with `crew_done`.
+- Overflow recovery tracks `agent_end`, `compaction_start/end`, and `auto_retry_start/end` events around `session.prompt()`.
+- State is in-memory only; subagent session files remain for post-hoc `/resume` inspection.
+
+### Current `pi-crew`
+
+- Primary runtime is child Pi process execution with durable `.crew/state` manifests and artifacts.
+- It has workflow/team abstractions, task graphs, worktree support, event log, mailbox, dashboard panes, render scheduler, notifications, and diagnostic exports.
+- It already has `live-session-runtime.ts`, but the current product surface centers on durable child-process workers rather than interactive in-process subagents.
+
+## Extension API Patterns Worth Reusing
+
+| Pattern | Reference source | Why it matters for current `pi-crew` |
+|---|---|---|
+| Owner-routed delivery by `sessionManager.getSessionId()` | `delivery-coordinator.ts` | Avoids sending async worker results to the wrong active session after `/resume`, `/new`, `/fork`, or multi-session use. |
+| Idle vs streaming delivery split | `subagent-messages.ts`, `delivery-coordinator.ts` | Prevents messages from getting stuck: idle sessions need `triggerTurn`; streaming sessions need `deliverAs: "steer"`. |
+| Deferred pending flush via `setTimeout(0)` | `delivery-coordinator.ts` | Avoids lost JSONL/custom-message persistence during resume before listeners reconnect. |
+| `extensionsOverride` filter | `bootstrap-session.ts` | Required for any in-process worker lane to prevent recursive subagent spawning. |
+| Fire-and-forget interactive response | `crew-respond.ts`, `crew-runtime.ts` | Lets parent stay responsive while an interactive worker continues in background. |
+| No duplicate done message | `crew-done.ts` | Avoids repeating the last subagent response during cleanup. |
+| Source-specific abort reasons | `crew-abort.ts`, `index.ts` shutdown handlers | Better diagnostics than generic "aborted by user". |
+| Emergency unrestricted abort command | `register-command.ts` | Useful escape hatch distinct from owner-scoped tool actions. |
+| Overflow tracker around SDK prompt | `overflow-recovery.ts` | Better UX for context overflow/compaction/retry in session-native workers. |
+
+## Key Differences / Non-Goals
+
+| Dimension | Reference `source/pi-crew` | Current `pi-crew` |
+|---|---|---|
+| Runtime | In-process `AgentSession` | Child Pi processes + durable orchestration |
+| State | In-memory map | Durable manifests/event logs/artifacts |
+| Scope | Flat subagent spawn/respond/done | Teams, workflows, task graph, worktrees |
+| Result UX | Pi steering/custom messages | Tool results, mailbox, dashboard, async status |
+| Interactive workers | Native | Not yet first-class |
+| Worktree isolation | None | First-class |
+| Replay/restart | Limited | Strong durable recovery |
+
+Do **not** replace the current runtime wholesale. Reference `source/pi-crew` lacks durable state, worktrees, workflow scheduling, artifact indexing, and the Phase 8 operator experience. Its best value is a narrower session-native execution lane and delivery correctness patterns.
+
+## Recommendations
+
+### P0 — Adopt Delivery Semantics for Async/Live Results
+
+Implement or adapt a small owner-routed delivery coordinator in current `pi-crew`:
+
+- Key by owner `sessionId`, not session file.
+- Queue pending messages when owner inactive.
+- On `session_start`, flush pending messages on next macrotask.
+- Use idle/streaming split:
+  - idle: `sendMessage(payload, { triggerTurn: true })`
+  - streaming: `sendMessage(payload, { deliverAs: "steer", triggerTurn: true })`
+- Keep current mailbox/event-log as durable source of truth; use delivery coordinator only for live UX.
+
+Likely target files:
+
+- `pi-crew/src/extension/register.ts`
+- `pi-crew/src/runtime/subagent-manager.ts`
+- `pi-crew/src/runtime/live-session-runtime.ts`
+- `pi-crew/src/extension/notification-router.ts`
+
+### P1 — Add Optional Session-Native Subagent Lane
+
+Build an opt-in lane on top of existing `live-session-runtime.ts` rather than changing the default child-process runtime:
+
+- `runtime.mode = "child-process" | "live-session" | "auto"` already exists conceptually; tighten semantics.
+- Use `SessionManager.newSession({ parentSession })` and `createAgentSession()` for in-process workers.
+- Filter `pi-crew` out of subagent resource loader extensions.
+- Persist minimal metadata to existing `.crew/state` so dashboards/recovery still work.
+
+This can reduce process startup overhead and blank console issues, while preserving child-process isolation as the safe default.
+
+### P1 — Introduce Interactive Worker Semantics
+
+Add first-class interactive subagents without disrupting teams:
+
+- New status: `waiting` for interactive background workers.
+- `crew_agent_respond` / `crew_agent_done` or extend existing `crew_agent_steer` semantics.
+- Fire-and-forget response: parent tool returns immediately; worker response arrives as mailbox/steering message.
+- `done` performs cleanup only; no duplicate response.
+
+Likely target files:
+
+- `pi-crew/src/runtime/crew-agent-records.ts`
+- `pi-crew/src/runtime/subagent-manager.ts`
+- `pi-crew/src/extension/registration/subagent-tools.ts`
+- `pi-crew/src/state/mailbox.ts`
+- `pi-crew/src/ui/dashboard-panes/agents-pane.ts`
+
+### P2 — Port Overflow Recovery Tracker for Live Sessions
+
+For session-native workers, wrap `AgentSession.prompt()` with an event tracker similar to `source/pi-crew/extension/runtime/overflow-recovery.ts`:
+
+- Track `compaction_start/end` and `auto_retry_start/end`.
+- Report recovered context overflow separately from hard failure.
+- Emit durable event-log records and dashboard health hints.
+
+This should not apply to child Pi workers directly; they already have process/transcript supervision.
+
+### P2 — Improve Abort Reason Taxonomy
+
+Adopt explicit abort source reasons across all worker paths:
+
+- tool-triggered abort
+- command-triggered emergency abort
+- session quit cleanup
+- session replacement detach/deactivate
+- watchdog timeout
+- stale heartbeat kill
+
+This improves diagnostics, notification routing, and Phase 9 reliability work.
+
+## Risks
+
+- In-process sessions reduce OS/process isolation; failures or leaks may affect the parent Pi process.
+- `extensionsOverride` is mandatory; missing it risks recursive subagent spawning.
+- Pi SDK internals may shift; keep this lane optional and covered by integration tests.
+- Delivery semantics must not bypass durable mailbox/event log; live messages are convenience, not source of truth.
+- Interactive workers can linger in memory; require TTL/status visibility and explicit cleanup.
+
+## Suggested Follow-Up Plan
+
+1. Write a focused design doc: `docs/research-session-native-runtime-plan.md`.
+2. Spike delivery coordinator only; no runtime swap.
+3. Add tests for idle/streaming/inactive owner delivery behavior.
+4. Add optional `live-session` worker lane behind config.
+5. Add interactive worker status/actions after live delivery is stable.
+
+## Research Artifacts
+
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/01_discover.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/02_explore-shard-1.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/03_explore-shard-2.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/results/04_explore-shard-3.txt`
+- `D:/my/my_project/.crew/artifacts/team_20260429091311_8047706b/batches/01_discover+02_explore-shard-1+03_explore-shard-2+04_explore-shard-3.md`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-crew",
-  "version": "0.1.34",
+  "version": "0.1.35",
   "description": "Pi extension for coordinated AI teams, workflows, worktrees, and async task orchestration",
   "author": "baphuongna",
   "license": "MIT",

package/src/extension/register.ts

@@ -318,7 +318,7 @@ export function registerPiTeams(pi: ExtensionAPI): void {
 			}
 		};
 		renderScheduler = new RenderScheduler(pi.events, renderTick, {
-			fallbackMs: loadedConfig.config.ui?.dashboardLiveRefreshMs ?? 750,
+			fallbackMs: loadedConfig.config.ui?.dashboardLiveRefreshMs ?? 250,
 			onInvalidate: () => getRunSnapshotCache(ctx.cwd).invalidate(),
 		});
 	});

package/src/runtime/child-pi.ts

@@ -430,7 +430,11 @@ export async function runChildPi(input: ChildPiRunInput): Promise<ChildPiRunResu
 					clearHardKillTimer(child.pid);
 				}
 				const timeoutError = responseTimeoutHit && !stderr.trim() ? { error: `Child Pi produced no new output for ${responseTimeoutMs}ms; process was terminated as unresponsive.` } : undefined;
-				settle({ exitCode, stdout, stderr, ...(forcedFinalDrain && !stderr.trim() ? { error: `Child Pi did not exit within ${finalDrainMs}ms after final assistant message; termination was requested.` } : {}), ...(timeoutError ? { error: timeoutError.error } : {}) });
+				// A final assistant event is the child Pi contract for "the worker produced its answer".
+				// Some Pi processes can linger during post-final cleanup/stdio shutdown; finalDrain terminates
+				// that lingering process so the parent can continue, but it must not turn a completed
+				// subagent answer into a failed task. Real pre-final response timeouts still report errors.
+				settle({ exitCode: forcedFinalDrain && !timeoutError ? 0 : exitCode, stdout, stderr, ...(timeoutError ? { error: timeoutError.error } : {}) });
 			});
 		});
 	} finally {

package/src/ui/dashboard-panes/agents-pane.ts

@@ -1,5 +1,7 @@
 import type { RunDashboardOptions } from "../run-dashboard.ts";
+import { iconForStatus } from "../status-colors.ts";
 import type { RunUiSnapshot } from "../snapshot-types.ts";
+import { spinnerFrame } from "../spinner.ts";
 
 function tokens(agent: RunUiSnapshot["agents"][number]): string {
 	const total = (agent.usage?.input ?? 0) + (agent.usage?.output ?? agent.progress?.tokens ?? 0) + (agent.usage?.cacheRead ?? 0) + (agent.usage?.cacheWrite ?? 0);
@@ -19,7 +21,8 @@ export function renderAgentsPane(snapshot: RunUiSnapshot | undefined, options: R
 				options.showTokens !== false ? tokens(agent) : undefined,
 				options.showModel !== false ? (agent.model ? `model=${agent.model}` : undefined) : undefined,
 			].filter((part): part is string => Boolean(part));
-			return `${agent.taskId} ${agent.role}->${agent.agent} · ${parts.join(" · ")}`;
+			const icon = iconForStatus(agent.status, { runningGlyph: spinnerFrame(agent.taskId) });
+			return `${icon} ${agent.taskId} ${agent.role}->${agent.agent} · ${parts.join(" · ")}`;
 		}),
 	];
 }

package/src/ui/live-run-sidebar.ts

@@ -13,9 +13,9 @@ import type { CrewTheme } from "./theme-adapter.ts";
 import { asCrewTheme, subscribeThemeChange } from "./theme-adapter.ts";
 import { Box, Text } from "./layout-primitives.ts";
 import type { RunSnapshotCache, RunUiSnapshot } from "./snapshot-types.ts";
+import { spinnerBucket, spinnerFrame } from "./spinner.ts";
 
 const TASK_READ_TTL_MS = 200;
-const SPINNER = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"];
 
 function renderLines(lines: string[], width: number): string[] {
 	const box = new Box(0, 0);
@@ -75,10 +75,11 @@ export class LiveRunSidebar {
 	}
 
 	private buildSignature(manifestStatus: string, tasks: TeamTaskState[], agents: ReturnType<typeof readCrewAgents>, waitingCount: number, snapshot?: RunUiSnapshot): string {
-		if (snapshot) return `${snapshot.signature}:${waitingCount}`;
+		const animation = agents.some((agent) => agent.status === "running") ? `:spin=${spinnerBucket()}` : "";
+		if (snapshot) return `${snapshot.signature}:${waitingCount}${animation}`;
 		const taskSig = tasks.map((task) => `${task.id}:${task.status}:${task.startedAt ?? ""}:${task.finishedAt ?? ""}:${task.agentProgress?.currentTool ?? ""}:${task.agentProgress?.toolCount ?? 0}:${task.agentProgress?.tokens ?? 0}:${task.usage ? JSON.stringify(task.usage) : ""}`).join("|");
 		const agentSig = agents.map((agent) => [agent.id, agent.status, agent.startedAt, agent.completedAt ?? "", agent.progress?.currentTool ?? "", agent.progress?.toolCount ?? 0, agent.progress?.tokens ?? 0, agent.progress?.turns ?? 0, agent.progress?.lastActivityAt ?? "", agent.progress?.recentOutput.at(-1) ?? "", agent.toolUses ?? 0].join(":")).join("|");
-		return `${manifestStatus}|${agents.length}|${waitingCount}|${taskSig}|${agentSig}`;
+		return `${manifestStatus}|${agents.length}|${waitingCount}|${taskSig}|${agentSig}${animation}`;
 	}
 
 	private colorLine(line: string): string {
@@ -140,7 +141,7 @@ export class LiveRunSidebar {
 				line(`Active agents (${active.length})`, w),
 			];
 			for (const agent of active.slice(0, 8)) {
-				const status = iconForStatus(agent.status, { runningGlyph: SPINNER[0] });
+				const status = iconForStatus(agent.status, { runningGlyph: spinnerFrame(agent.taskId) });
 				const usage = agent.usage ? formatUsage(agent.usage) : agent.progress?.tokens ? `tokens=${agent.progress.tokens}` : "usage=pending";
 				lines.push(line(`${status} ${agent.taskId} ${agent.role}->${agent.agent}`, w));
 				lines.push(line(`  ${agent.routing ? `model ${agent.routing.requested ? `${agent.routing.requested} → ` : ""}${agent.routing.resolved}` : agent.model ? `model ${agent.model}` : "model pending"}`, w));

package/src/ui/run-dashboard.ts

@@ -19,6 +19,7 @@ import { renderTranscriptPane } from "./dashboard-panes/transcript-pane.ts";
 import { renderHealthPane } from "./dashboard-panes/health-pane.ts";
 import { dashboardActionForKey } from "./keybinding-map.ts";
 import type { RunSnapshotCache, RunUiSnapshot } from "./snapshot-types.ts";
+import { spinnerBucket, spinnerFrame } from "./spinner.ts";
 
 interface DashboardComponent {
 	invalidate(): void;
@@ -140,7 +141,8 @@ function agentPreviewLine(agent: CrewAgentRecord, task: TeamTaskState | undefine
 		agent.startedAt ? `age=${formatAge(agent.completedAt ?? agent.startedAt)}` : undefined,
 	].filter((part): part is string => Boolean(part));
 	const recent = agent.progress?.recentOutput?.at(-1);
-	return `Agent: ${iconForStatus(agent.status)} ${agent.taskId} ${agent.role}->${agent.agent}${stats.length ? ` · ${stats.join(" · ")}` : ""}${recent ? ` ⎿ ${recent}` : ""}`;
+	const icon = iconForStatus(agent.status, { runningGlyph: spinnerFrame(agent.taskId) });
+	return `Agent: ${icon} ${agent.taskId} ${agent.role}->${agent.agent}${stats.length ? ` · ${stats.join(" · ")}` : ""}${recent ? ` ⎿ ${recent}` : ""}`;
 }
 
 function readAgentPreview(run: TeamRunManifest, maxLines = 5, options: RunDashboardOptions = {}): string[] {
@@ -189,7 +191,7 @@ function runLabel(run: TeamRunManifest, selected: boolean, snapshotCache?: RunSn
 	const step = stale ? "orphaned queued run" : running ? `step ${running.taskId}` : queued ? `queued ${queued.taskId}` : `agents ${agents.length}`;
 	const status: RunStatus = stale ? "stale" : (run.status as RunStatus);
 	const marker = selected ? "›" : " ";
-	return `${marker} ${iconForStatus(status)} ${run.runId.slice(-8)} ${status} | ${run.team}/${run.workflow ?? "none"} | ${step} | ${run.goal}`;
+	return `${marker} ${iconForStatus(status, { runningGlyph: spinnerFrame(run.runId) })} ${run.runId.slice(-8)} ${status} | ${run.team}/${run.workflow ?? "none"} | ${step} | ${run.goal}`;
 }
 
 function groupedRuns(runs: TeamRunManifest[], snapshotCache?: RunSnapshotCache): Array<{ label: string; run?: TeamRunManifest }> {
@@ -253,15 +255,17 @@ export class RunDashboard implements DashboardComponent {
 	}
 
 	private buildSignature(): string {
+		let hasRunning = false;
 		const statuses = this.runs.map((run) => {
 			const snapshot = snapshotFor(run, this.options.snapshotCache);
 			const displayRun = snapshot?.manifest ?? run;
 			const agents = snapshot?.agents ?? agentsFor(run, this.options.snapshotCache);
 			const stale = isLikelyOrphanedActiveRun(displayRun, agents);
 			const status: RunStatus = stale ? "stale" : (displayRun.status as RunStatus);
+			if (status === "running" || agents.some((agent) => agent.status === "running")) hasRunning = true;
 			return snapshot?.signature ?? `${displayRun.runId}:${displayRun.status}:${displayRun.updatedAt}:${status}`;
 		}).join("|");
-		return `${this.selected}:${this.showFullProgress ? 1 : 0}:${this.activePane}:${statuses}`;
+		return `${this.selected}:${this.showFullProgress ? 1 : 0}:${this.activePane}:${statuses}${hasRunning ? `:spin=${spinnerBucket()}` : ""}`;
 	}
 
 	invalidate(): void {

package/src/ui/spinner.ts

@@ -0,0 +1,17 @@
+export const SUBAGENT_SPINNER_FRAMES = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"] as const;
+export const SUBAGENT_SPINNER_FRAME_MS = 160;
+
+export function spinnerBucket(now = Date.now(), frameMs = SUBAGENT_SPINNER_FRAME_MS): number {
+	return Math.floor(now / Math.max(1, frameMs));
+}
+
+function hashKey(key: string): number {
+	let hash = 0;
+	for (let index = 0; index < key.length; index += 1) hash = (hash * 31 + key.charCodeAt(index)) >>> 0;
+	return hash;
+}
+
+export function spinnerFrame(key = "", now = Date.now()): string {
+	const offset = key ? hashKey(key) % SUBAGENT_SPINNER_FRAMES.length : 0;
+	return SUBAGENT_SPINNER_FRAMES[(spinnerBucket(now) + offset) % SUBAGENT_SPINNER_FRAMES.length] ?? SUBAGENT_SPINNER_FRAMES[0];
+}