@gotgenes/pi-subagents 10.2.1 → 11.0.1

package/docs/retro/0229-agent-born-complete.md

@@ -0,0 +1,89 @@
+---
+issue: 229
+issue_title: "Agent born complete: Agent.run() absorbs startAgent (Phase 15, Step 4)"
+---
+
+# Retro: #229 — Agent born complete: Agent.run() absorbs startAgent
+
+## Stage: Planning (2026-05-27T18:00:00Z)
+
+### Session summary
+
+Produced a 9-step TDD plan for absorbing `AgentManager.startAgent()` into `Agent.run()`.
+Key design decisions: per-agent `AgentLifecycleObserver` interface passed at construction (chosen over callback fields and EventEmitter), and fully async worktree error surface (chosen over split sync/async).
+
+### Observations
+
+- **Observer pattern chosen over callbacks:** The per-agent `AgentLifecycleObserver` interface replaces three separate mechanisms (`onSessionCreated` callback, `setOnRunFinished`, `onCompact` callback).
+  All methods are optional, composed by `AgentManager.buildObserver()` per spawn.
+- **`ParentSessionInfo`/`CompactionInfo` relocation needed:** `agent.ts` importing from `agent-manager.ts` would create a circular type import (agent-manager already imports `Agent`).
+  Moving both types to `types.ts` in step 1 avoids the cycle.
+- **`AgentInit` grows wide (15+ optional fields):** Making run-config fields optional preserves backward compat for the 55+ `new Agent()` calls in tests.
+  Noted as a known smell — follow-up issues (#230 ConcurrencyQueue, potential `AgentInit` restructuring) may address this.
+- **Async error surface changes tool behavior:** `background-spawner.ts`'s try/catch around `manager.spawn()` becomes unreachable for worktree errors.
+  Keeping it for robustness; the error surfaces on `record.error` instead.
+- **Lift-and-shift TDD order:** Steps 3–5 incrementally change `AgentInit`, `setupWorktree`, and `completeRun`/`failRun` before step 6 adds `Agent.run()`.
+  This avoids a single massive step that rewrites everything at once.
+
+## Stage: Implementation — TDD (2026-05-28T01:00:00Z)
+
+### Session summary
+
+Completed all 9 TDD steps in 9 commits (plus 2 planning/retro docs commits).
+Test count went from 1005 to 1020 (+15 tests).
+`AgentManager.startAgent()`, `SpawnArgs`, and `onSessionCreated` callback are deleted.
+`Agent.run()` now owns the full execution lifecycle.
+
+### Observations
+
+- **Steps 7–8 merged in practice:** The tool-layer `onSessionCreated` → `observer` migration (step 8) had to be done alongside the `AgentSpawnConfig` change (step 7) because removing the `onSessionCreated` field broke compilation of `background-spawner.ts` and `foreground-runner.ts`.
+  This was expected — they share the same type.
+- **`setupWorktree` kept public:** The plan called for making it private in step 4, but it was kept public through step 6 since the manager still called it.
+  After step 7 (Agent.run() absorbs the call), it could be made private; left as a minor follow-up (reviewer flagged as WARN).
+- **`isBackground` removed from Agent storage:** The field was declared on `AgentInit` but Agent never reads it — the manager resolves `isBackground` before construction (setting initial status and composing the observer).
+  Biome flagged it as unused; removed from stored fields, kept on `AgentInit` for the manager's use.
+- **Worktree error surface confirmed async:** The `agent-manager.test.ts` test for synchronous worktree throw was rewritten to verify the error surfaces on `record.error` after awaiting the promise.
+  `background-spawner.ts` try/catch around `spawn()` retained for robustness.
+- **Pre-completion reviewer:** WARN — 3 non-blocking findings: `setupWorktree` not marked private, `isBackground` dead field on `AgentInit`, and `package-pi-subagents` SKILL.md Phase 15 description referencing deleted `startAgent`.
+
+## Stage: Final Retrospective (2026-05-28T01:30:00Z)
+
+### Session summary
+
+Planned, implemented (9 TDD steps), shipped, and released `pi-subagents-v11.0.0` in a single session spanning planning → TDD → ship → retro.
+Test count: 1005 → 1020 (+15).
+Also filed #249 (`pi-permission-system` bash external-directory gate bug) discovered during the pre-completion review.
+
+### Observations
+
+#### What went well
+
+- The lift-and-shift TDD strategy (steps 1–5 incrementally preparing, step 6 adding `Agent.run()`, step 7 rewriting `spawn()`) kept every commit compilable and green.
+  No step required backtracking.
+- The `ask-user` call during planning (observer pattern vs callbacks, sync vs async error surface) front-loaded design decisions that would have caused rework if deferred.
+- Pre-commit hooks caught both lint failures (`no-unnecessary-condition` on `abortController?.abort()`, `unbound-method` on observer forwarding) before they reached CI.
+
+#### What caused friction (agent side)
+
+- `wrong-abstraction` — The plan separated step 7 (remove `onSessionCreated` from `AgentSpawnConfig`) and step 8 (update tool-layer consumers) as distinct commits.
+  Removing the field immediately broke `background-spawner.ts` and `foreground-runner.ts` at compilation, forcing a merge.
+  Impact: added friction but no rework — the merge was straightforward since both files needed the same `onSessionCreated` → `observer` transformation.
+  Added a testing skill rule to catch this pattern in future plans.
+
+#### What caused friction (user side)
+
+- The pre-completion reviewer's Mermaid validation (`mmdc -o /tmp/mermaid-check.svg`) triggered a permission prompt from `pi-permission-system` despite `/tmp/*` being configured as `"allow"` in the global config.
+  This was a genuine bug (#249) in the bash external-directory gate, not a config mistake.
+  The prompt interrupted the automated review flow, requiring manual approval.
+
+### Diagnostic details
+
+- **Model-performance correlation** — Pre-completion reviewer ran on `claude-sonnet-4-6-20260526`; appropriate for judgment-heavy review work (code design, acceptance criteria, mermaid validation).
+  No mismatches detected.
+- **Feedback-loop gap analysis** — `pnpm run check` and `pnpm run test` were run after every TDD step commit, not just at the end.
+  `pnpm run lint` ran at the end (post-TDD checks) and at pre-push, which is correct since lint is slower and pre-commit hooks catch most issues incrementally.
+
+### Changes made
+
+1. `.pi/skills/testing/SKILL.md` — added TDD planning rule: when a step removes a field from a shared interface, all downstream readers must update in the same step.
+2. `packages/pi-subagents/docs/retro/0229-agent-born-complete.md` — appended Final Retrospective stage entry.

package/docs/retro/0231-push-exec-registry-to-runner.md

@@ -38,3 +38,34 @@ Pre-completion reviewer returned PASS.
   Fixed as a lint cleanup in the doc commit.
 - The `sed`-based bulk replacement for `runAgent(..., io)` → `runAgent(..., { io, exec, registry: mockAgentLookup })` missed one multi-line call site (the `rejects.toThrow` test wrapping the call in `expect()`).
   Caught immediately by the test run.
+
+## Stage: Final Retrospective (2026-05-27T22:43:52Z)
+
+### Session summary
+
+Shipped #231 cleanly: CI passed on first push, issue closed, release `pi-subagents-v10.2.1` published.
+The entire issue (plan → TDD → ship) completed in one sitting with no user intervention needed.
+
+### Observations
+
+#### What went well
+
+- The `RunnerDeps` design was unambiguous — the `ask_user` gate in planning correctly identified the one genuine design choice (`RunContext` fate) and got user input before proceeding.
+- Pre-completion reviewer returned PASS with zero findings, confirming the mechanical refactoring was clean.
+- Merging plan steps 3–5 during TDD was the right call; the testing skill rule about single-call-site interfaces caught the plan's error before any broken commit landed.
+
+#### What caused friction (agent side)
+
+- `wrong-abstraction` — The plan listed steps 3, 4, and 5 as separate commits and claimed "each commit is independently valid," but removing fields from `RunContext` (step 3) immediately caused TypeScript excess-property errors in `AgentManager` (step 4) and `index.ts` (step 5).
+  The existing `/plan-issue` rule (line 109) covers removing exports with single call sites, but did not trigger recognition because this was *shrinking* an interface, not removing one.
+  Impact: the TDD agent had to merge three steps on the fly — no rework, but the plan was misleading.
+- `missing-context` — The `sed`-based bulk replacement for `runAgent(..., io)` missed one multi-line call site where `}, io)` appeared on a different line than the opening `runAgent(`.
+  Impact: one extra manual edit; caught immediately by the test run.
+
+#### What caused friction (user side)
+
+- No friction observed — the user's involvement was limited to confirming the `RunContext` design choice during planning.
+
+### Changes made
+
+1. `.pi/prompts/plan-issue.md` — added a rule under TDD Order: when a step removes fields from an interface, include downstream object-literal call-site updates in the same step (TypeScript excess property checking).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gotgenes/pi-subagents",
-  "version": "10.2.1",
+  "version": "11.0.1",
   "type": "module",
   "exports": {
     ".": "./src/service.ts"

package/src/lifecycle/agent-manager.ts

@@ -8,26 +8,22 @@
 
 import { randomUUID } from "node:crypto";
 import type { Model } from "@earendil-works/pi-ai";
-import type { AgentSession } from "@earendil-works/pi-coding-agent";
 import { debugLog } from "#src/debug";
-import { Agent } from "#src/lifecycle/agent";
+import { Agent, type AgentLifecycleObserver } from "#src/lifecycle/agent";
 import type { AgentRunner } from "#src/lifecycle/agent-runner";
 import type { ParentSnapshot } from "#src/lifecycle/parent-snapshot";
 import type { WorktreeManager } from "#src/lifecycle/worktree";
 
-import { NotificationState } from "#src/observation/notification-state";
 import { subscribeAgentObserver } from "#src/observation/record-observer";
 import type { RunConfig } from "#src/runtime";
-import type { AgentInvocation, IsolationMode, SubagentType, ThinkingLevel } from "#src/types";
-
-export type CompactionInfo = { reason: "manual" | "threshold" | "overflow"; tokensBefore: number };
+import type { AgentInvocation, CompactionInfo, IsolationMode, ParentSessionInfo, SubagentType, ThinkingLevel } from "#src/types";
 
 /** Observer interface for agent lifecycle notifications. */
 export interface AgentManagerObserver {
   onAgentStarted(record: Agent): void;
   onAgentCompleted(record: Agent): void;
   onAgentCompacted(record: Agent, info: CompactionInfo): void;
-  /** Fires synchronously after a background agent record is created (before startAgent). */
+  /** Fires synchronously after a background agent record is created (before run). */
   onAgentCreated(record: Agent): void;
 }
 
@@ -43,22 +39,6 @@ export interface AgentManagerOptions {
   observer?: AgentManagerObserver;
 }
 
-interface SpawnArgs {
-  snapshot: ParentSnapshot;
-  type: SubagentType;
-  prompt: string;
-  options: AgentSpawnConfig;
-}
-
-export interface ParentSessionInfo {
-  /** Path to the parent session's JSONL file (for deriving the subagent session directory). */
-  parentSessionFile?: string;
-  /** Session ID of the parent agent (stored in the child session's parentSession header). */
-  parentSessionId?: string;
-  /** Tool call ID for background notification wiring. When set, spawn attaches NotificationState. */
-  toolCallId?: string;
-}
-
 export interface AgentSpawnConfig {
   description: string;
   model?: Model<any>;
@@ -79,8 +59,8 @@ export interface AgentSpawnConfig {
   invocation?: AgentInvocation;
   /** Parent abort signal - when aborted, the subagent is also stopped. */
   signal?: AbortSignal;
-  /** Called when the agent session is created - receives the session and the agent's record. */
-  onSessionCreated?: (session: AgentSession, record: Agent) => void;
+  /** Per-agent lifecycle observer — replaces onSessionCreated callback. */
+  observer?: AgentLifecycleObserver;
   /** Parent session identity - grouped fields that travel together from the tool boundary. */
   parentSession?: ParentSessionInfo;
 }
@@ -94,8 +74,8 @@ export class AgentManager {
   private readonly _getMaxConcurrent: () => number;
   private getRunConfig?: () => RunConfig;
 
-  /** Queue of background agents waiting to start. */
-  private queue: { id: string; args: SpawnArgs }[] = [];
+  /** Queue of background agent IDs waiting to start. */
+  private queue: string[] = [];
   /** Number of currently running background agents. */
   private runningBackground = 0;
   constructor(options: AgentManagerOptions) {
@@ -117,6 +97,25 @@ export class AgentManager {
     this.drainQueue();
   }
 
+  /** Compose a per-agent lifecycle observer from manager and spawn-config concerns. */
+  private buildObserver(options: AgentSpawnConfig): AgentLifecycleObserver {
+    return {
+      onStarted: (agent) => {
+        if (options.isBackground) this.runningBackground++;
+        this.observer?.onAgentStarted(agent);
+      },
+      onSessionCreated: options.observer?.onSessionCreated
+        ? (agent, session) => options.observer!.onSessionCreated!(agent, session)
+        : undefined,
+      onRunFinished: (agent) => {
+        if (options.isBackground) this.finalizeBackgroundRun(agent);
+      },
+      onCompacted: (agent, info) => {
+        this.observer?.onAgentCompacted(agent, info);
+      },
+    };
+  }
+
   /**
    * Spawn an agent and return its ID immediately (for background use).
    * If the concurrency limit is reached, the agent is queued.
@@ -128,90 +127,45 @@ export class AgentManager {
     options: AgentSpawnConfig,
   ): string {
     const id = randomUUID().slice(0, 17);
-    const abortController = new AbortController();
     const record = new Agent({
       id,
       type,
       description: options.description,
       status: options.isBackground ? "queued" : "running",
       startedAt: Date.now(),
-      abortController,
       invocation: options.invocation,
+      // Run config
+      snapshot,
+      prompt,
+      model: options.model,
+      maxTurns: options.maxTurns,
+      isolated: options.isolated,
+      thinkingLevel: options.thinkingLevel,
+      isolation: options.isolation,
+      parentSession: options.parentSession,
+      signal: options.signal,
+      // Shared deps
+      runner: this.runner,
+      worktrees: this.worktrees,
+      observer: this.buildObserver(options),
+      getRunConfig: this.getRunConfig,
     });
     this.agents.set(id, record);
 
-    if (options.parentSession?.toolCallId) {
-      record.notification = new NotificationState(options.parentSession.toolCallId);
-    }
-
     if (options.isBackground) {
       this.observer?.onAgentCreated(record);
     }
 
-    const args: SpawnArgs = { snapshot, type, prompt, options };
-
     if (options.isBackground && !options.bypassQueue && this.runningBackground >= this._getMaxConcurrent()) {
       // Queue it - will be started when a running agent completes
-      this.queue.push({ id, args });
+      this.queue.push(id);
       return id;
     }
 
-    // setupWorktree can throw (e.g. strict worktree-isolation failure) - clean
-    // up the record so callers don't see an orphan in `listAgents()`.
-    try {
-      record.setupWorktree(this.worktrees, options.isolation);
-      record.promise = this.startAgent(id, record, args);
-    } catch (err) {
-      this.agents.delete(id);
-      throw err;
-    }
+    record.promise = record.run();
     return id;
   }
 
-  /** Actually start an agent (called immediately or from queue drain). */
-  private async startAgent(id: string, record: Agent, { snapshot, type, prompt, options }: SpawnArgs): Promise<void> {
-    record.markRunning(Date.now());
-    if (options.isBackground) this.runningBackground++;
-    this.observer?.onAgentStarted(record);
-
-    record.setOnRunFinished(
-      options.isBackground ? () => this.finalizeBackgroundRun(record) : undefined,
-    );
-    record.wireSignal(options.signal, () => this.abort(id));
-
-    const runConfig = this.getRunConfig?.();
-    try {
-      const result = await this.runner.run(snapshot, type, prompt, {
-        context: {
-          cwd: record.worktreeState?.path,
-          parentSession: options.parentSession,
-        },
-        model: options.model,
-        maxTurns: options.maxTurns,
-        defaultMaxTurns: runConfig?.defaultMaxTurns,
-        graceTurns: runConfig?.graceTurns,
-        isolated: options.isolated,
-        thinkingLevel: options.thinkingLevel,
-        signal: record.abortController!.signal,
-        onSessionCreated: (session) => {
-          // Capture the session file path early so it's available for display
-          // before the run completes (e.g. in background agent status messages).
-          // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- sessionManager is typed as always present but Pi SDK may not provide it
-          const outputFile = session.sessionManager?.getSessionFile?.() ?? undefined;
-          record.execution = { session, outputFile };
-          record.flushPendingSteers(session);
-          record.attachObserver(subscribeAgentObserver(session, record, {
-            onCompact: (r, info) => this.observer?.onAgentCompacted(r, info),
-          }));
-          options.onSessionCreated?.(session, record);
-        },
-      });
-      record.completeRun(result, this.worktrees);
-    } catch (err) {
-      record.failRun(err, this.worktrees);
-    }
-  }
-
   /** Decrement background counter, notify observer (crash-safe), and drain the queue. */
   private finalizeBackgroundRun(record: Agent): void {
     this.runningBackground--;
@@ -222,18 +176,10 @@ export class AgentManager {
   /** Start queued agents up to the concurrency limit. */
   private drainQueue() {
     while (this.queue.length > 0 && this.runningBackground < this._getMaxConcurrent()) {
-      const next = this.queue.shift()!;
-      const record = this.agents.get(next.id);
+      const id = this.queue.shift()!;
+      const record = this.agents.get(id);
       if (record?.status !== "queued") continue;
-      try {
-        record.setupWorktree(this.worktrees, next.args.options.isolation);
-        record.promise = this.startAgent(next.id, record, next.args);
-      } catch (err) {
-        // Late failure (e.g. strict worktree-isolation) - surface on the record
-        // so the user/agent can see it via /agents, then keep draining.
-        record.markError(err);
-        this.observer?.onAgentCompleted(record);
-      }
+      record.promise = record.run();
     }
   }
 
@@ -301,7 +247,7 @@ export class AgentManager {
 
     // Remove from queue if queued
     if (record.status === "queued") {
-      this.queue = this.queue.filter(q => q.id !== id);
+      this.queue = this.queue.filter(qid => qid !== id);
       record.markStopped();
       return true;
     }
@@ -349,8 +295,8 @@ export class AgentManager {
   abortAll(): number {
     let count = 0;
     // Clear queued agents first
-    for (const queued of this.queue) {
-      const record = this.agents.get(queued.id);
+    for (const id of this.queue) {
+      const record = this.agents.get(id);
       if (record) {
         record.markStopped();
         count++;

package/src/lifecycle/agent-runner.ts

@@ -9,14 +9,13 @@ import {
   type SettingsManager,
 } from "@earendil-works/pi-coding-agent";
 import type { AgentConfigLookup } from "#src/config/agent-types";
-import type { ParentSessionInfo } from "#src/lifecycle/agent-manager";
 import type { ParentSnapshot } from "#src/lifecycle/parent-snapshot";
 import { registerChildSession, unregisterChildSession } from "#src/lifecycle/permission-bridge";
 import { extractAssistantContent } from "#src/session/content-items";
 import { extractText } from "#src/session/context";
 import type { EnvInfo } from "#src/session/env";
 import { type AssemblerIO, assembleSessionConfig } from "#src/session/session-config";
-import type { ShellExec, SubagentType, ThinkingLevel } from "#src/types";
+import type { ParentSessionInfo, ShellExec, SubagentType, ThinkingLevel } from "#src/types";
 
 /** Names of tools registered by this extension that subagents must NOT inherit. */
 const EXCLUDED_TOOL_NAMES = ["subagent", "get_subagent_result", "steer_subagent"];