@zachwill/pi-orchestrate 0.9.2 → 0.11.0

package/README.md

@@ -1,125 +1,53 @@
 # Pi Orchestrate
 
-[`@zachwill/pi-orchestrate`](https://www.npmjs.com/package/@zachwill/pi-orchestrate) lets a Pi agent delegate work to separate child sessions, run independent tasks concurrently, and synthesize their results.
+[`@zachwill/pi-orchestrate`](https://www.npmjs.com/package/@zachwill/pi-orchestrate) lets a Pi session delegate work to direct child sessions.
 
-## Install
+- Each worker gets a focused brief and a separate conversation.
+- Workers run independently and return their results to the parent.
+- Only the parent can delegate; workers cannot create more workers.
 
-```bash
-pi install npm:@zachwill/pi-orchestrate
-```
+## The model
 
-The package includes `scout`, `investigator`, `web`, and `worker` definitions. Pi packages and workers run with your system permissions, so review every package and worker definition you trust.
+Each `orchestrate` dispatch creates a fresh worker session with its own transcript. The worker receives a complete brief from the parent but not the parent's conversation.
 
-## How it works
+A worker definition is reusable configuration: it selects the worker's prompt, tools, lifecycle, and optional model settings. It is not a running or retained session.
 
-The current Pi session remains the parent and owns the task. Each `orchestrate` call starts a fresh worker session with its own transcript and a complete brief from the parent.
+Independent workers dispatched together run concurrently. Their results return only to the parent session that started them, and the parent synthesizes the group after every worker finishes. A rejected or failed worker does not cancel its peers. Orchestration dispatched alongside unrelated tool calls runs inline instead of in the background.
 
 Workers have one of two lifecycles:
 
-- `one-shot` workers return one result and stop automatically.
-- `interactive` workers return a result and remain `ready` for follow-up work.
-
-A **worker ID** identifies a worker session. A **run ID** identifies one generation within that session. Each interactive follow-up creates a new run while keeping the same worker ID.
-
-## Tools
+- A `one-shot` worker returns one result and stops.
+- An `interactive` worker returns a result and remains available for follow-up work.
 
-Pi Orchestrate adds exactly five tools.
+A **worker ID** identifies a worker session. A **run ID** identifies one generation within it. An interactive follow-up creates a new run while preserving the worker ID and prior session context.
 
-### `orchestrate`
-
-```text
-orchestrate({ worker, title, instructions })
-```
+Interactive workers remain available across session switches and extension reloads within the same Pi process. Closing one releases its retained session; process shutdown releases any that remain.
 
-Starts one worker. `title` is a short label; `instructions` is the complete, self-contained brief, including scope, constraints, success criteria, and expected output.
-
-A sole call runs asynchronously. To run independent work concurrently, send the complete wave as sibling calls in one assistant response:
-
-```text
-orchestrate({
-  worker: "investigator",
-  title: "Trace configuration loading",
-  instructions: "Trace configuration loading from entry point to runtime. Read only. Return the relevant symbols, file paths, and a concise data-flow summary."
-})
-
-orchestrate({
-  worker: "investigator",
-  title: "Trace shutdown cleanup",
-  instructions: "Trace shutdown and cleanup behavior. Read only. Return the relevant symbols, lifecycle invariants, and uncovered edge cases."
-})
-```
+## Agent interface
 
-A wave must contain all N intended `orchestrate` calls and no other tool calls. Form the whole wave before sending it; a successfully admitted asynchronous call ends the parent turn. When Pi provides a parallel tool dispatcher such as `multi_tool_use.parallel`, put all N `functions.orchestrate` entries in one dispatcher call. Otherwise, emit N native sibling calls.
+Pi Orchestrate gives the parent five model-facing tools:
 
-Pure orchestration groups run asynchronously, and their siblings execute concurrently. Mixing another tool into the group makes orchestration inline and blocking. Inline work follows cancellation of the parent turn; accepted asynchronous work continues independently.
+| Tool | Purpose |
+| --- | --- |
+| `orchestrate` | Start a fresh worker with a title and complete brief |
+| `interactive_send` | Continue an owned interactive worker that is ready |
+| `interactive_close` | Release an owned interactive worker that is ready |
+| `worker_abort` | Stop active workers owned by the parent |
+| `worker_status` | Inspect the trusted catalog and diagnose the parent's worker state |
 
-Each call validates its input, worker definition, and model before starting. Sibling calls are admitted independently, so one rejection or worker failure does not roll back its peers. There is no extension-level sibling-group limit or hidden throttle.
+The extension supplies the parent with the exact dispatch and lifecycle rules for these tools. The README describes their behavior rather than duplicating those model instructions.
 
-Asynchronous results return to the exact parent session automatically. A sibling wave produces one final parent synthesis turn after every admitted worker settles. If that parent is busy or inactive, results wait for it; they are never delivered to another session.
-
-### `worker_abort`
-
-```text
-worker_abort({ worker_ids: ["worker-…"] })
-worker_abort({ all: true })
-```
-
-Stops active workers owned by the current parent session. Explicit IDs are validated together. `{ all: true }` is a no-op when no owned workers are active and does not close interactive workers that are already `ready`.
-
-Completed one-shot workers need no cleanup. Abort active interactive work before closing its retained session.
-
-### `worker_status`
-
-```text
-worker_status({})
-```
+## Worker definitions
 
-Returns the trusted worker catalog, catalog diagnostics, and the current parent session's worker and run state. It omits worker prompts and full task instructions.
+The package includes four fallback definitions in [`examples/workers/`](examples/workers/): `scout` for small factual probes, `investigator` for read-only cross-file research, `worker` for bounded implementation, and `web` for public web research. The first three inherit the parent's active model. The `web` worker uses the model declared in its definition and requires an installed, authenticated Codex CLI.
 
-Use it for diagnostics and recovery, not completion polling. Normal asynchronous results arrive automatically. The TUI widget separately shows active work, and the footer reports interactive workers that are ready.
+Definitions are loaded by name in this precedence order:
 
-### `interactive_close`
-
-```text
-interactive_close({ worker_id: "worker-…" })
-```
-
-Closes an owned interactive worker whose status is `ready`. It releases the retained session; it cannot close active work or a one-shot worker.
-
-### `interactive_send`
-
-```text
-interactive_send({
-  worker_id: "worker-…",
-  instructions: "Verify the first risk against the tests and cite the relevant cases."
-})
-```
-
-Starts a follow-up run on an owned interactive worker whose status is `ready`. The worker keeps its ID and prior session context. Send `interactive_send` as the only tool call in the assistant message to run it asynchronously; sibling tool calls make it inline and blocking.
-
-Interactive workers remain available across extension reloads and session switches within the same Pi process. Close them when their continuity is no longer useful. Runtime shutdown releases retained sessions automatically.
-
-## Parent responsibilities
-
-The parent agent still owns the result:
-
-- Delegate bounded, independent scopes and materially distinct validation perspectives. Keep trivial or tightly coupled work in the parent.
-- Use as many workers as the task supports instead of a small fixed count. User-named roles and counts are a floor unless the user sets an exact cap.
-- Give each worker a complete brief and non-overlapping write scope. Intentional overlap should serve a distinct review perspective.
-- Dispatch each full wave together. As findings expose new independent work, dispatch another full wave.
-- Review the evidence and changes, resolve conflicts, verify the integrated result, and answer from the parent session.
-
-## Configure workers
-
-Worker definitions are loaded by name in this precedence order:
-
-1. Package fallbacks in [`examples/workers/`](examples/workers/)
+1. Package fallbacks in `examples/workers/`
 2. User definitions in `~/.pi/agent/pi-orchestrate/workers/*.md`
 3. Project definitions in `<project>/.pi/pi-orchestrate/workers/*.md`, when Pi trusts the project
 
-A later definition replaces an earlier definition with the same name. Untrusted projects contribute no project definitions.
-
-The bundled `scout`, `investigator`, and `worker` inherit the parent's active model. The bundled `web` worker requires an installed, authenticated Codex CLI and uses `openai-codex/gpt-5.6-sol`. Copy a fallback into a user or project directory to customize it.
+A later definition replaces an earlier definition with the same name. Untrusted projects contribute no project definitions. Copy a fallback into a user or project directory to replace it.
 
 A definition is a Markdown file whose basename matches its `name`:
 
@@ -134,20 +62,31 @@ lifecycle: interactive
 Inspect only the assigned scope. Do not modify files. Return concise findings with file paths.
 ```
 
-Required fields are `name`, `description`, and a nonempty `tools` list. The Markdown body is the worker's nonempty system prompt.
+The frontmatter is strict:
+
+| Field | Meaning |
+| --- | --- |
+| `name` | Required definition name; must match the filename |
+| `description` | Required catalog description used when choosing a worker |
+| `tools` | Required nonempty tool list |
+| `lifecycle` | `one-shot` or `interactive`; defaults to `one-shot` |
+| `model` | Optional `provider/model` coordinate; omission inherits the parent's model |
+| `thinking` | Optional thinking level: `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, or `max` |
+| `skills` | Optional exact skill allowlist; `[]` disables skills and omission uses normal discovery |
+| `compaction` | Optional Pi compaction settings: `enabled`, `reserveTokens`, and `keepRecentTokens` |
 
-`lifecycle` is optional, accepts `one-shot` or `interactive`, and defaults to `one-shot`. Other optional fields are `model`, `thinking`, `skills`, and `compaction`. An omitted `model` inherits the parent's model. For `skills`, omission uses normal discovery, a list is an exact allowlist, and `[]` disables skills.
+The Markdown body is the worker's nonempty system prompt. `tools` and `skills` accept either YAML arrays or comma-separated strings. Supported Pi tools are `read`, `bash`, `edit`, `write`, `grep`, `find`, and `ls`. Unknown fields and malformed definitions are rejected and appear in catalog diagnostics.
 
-Definitions are strict, regular non-symlink `.md` files up to 64 KiB. Supported Pi tools are `read`, `bash`, `edit`, `write`, `grep`, `find`, and `ls`. Grant the smallest useful set: a read-only prompt does not prevent writes when the worker has tools with write authority.
+Each `orchestrate` call starts a new worker session from the selected definition. Only `interactive_send` continues an existing session.
 
-A definition is reusable dispatch configuration, not a retained session. Each `orchestrate` call starts a new worker session; only `interactive_send` continues an existing one.
+## Trust boundary
 
-## Trust and isolation
+Workers run in the parent process and are not security sandboxes. They share its filesystem and environment permissions.
 
-Workers receive fresh durable Pi child-session lineage without the parent's conversation. They run in the parent process and are not security sandboxes: they share its filesystem and environment permissions.
+Workers can use global Pi settings, authentication, packages, extensions, skills, and context. Trusted projects may add project-scoped resources. Untrusted projects do not contribute project workers, settings, extensions, skills, or context; global resources remain available.
 
-Workers use global Pi settings, authentication, packages, extensions, skills, and context. Trusted projects may add project-scoped resources. Untrusted projects do not contribute project workers, settings, extensions, skills, or context; global resources remain available.
+A definition's `tools` field controls Pi's tool allowlist, not operating-system authority. A worker with `bash` can start external processes, including other agent CLIs. A read-only prompt also does not prevent writes when the worker has a write-capable tool.
 
-A definition's `tools` field controls Pi's tool allowlist, not operating-system authority. A worker with `bash` can start external processes, including agent CLIs. Give concurrent workers separate write scopes and inspect their changes in the parent.
+Concurrent workers share the same working tree, so overlapping write scopes can collide. The parent owns reviewing their output and integrating the result.
 
-Pi Orchestrate excludes itself from child sessions and instructs workers not to create descendant Pi worker sessions. Workers remain direct Pi children.
+Pi Orchestrate excludes itself from child sessions and keeps workers as direct Pi children.

package/extension/catalog/definition.ts

@@ -0,0 +1,89 @@
+import type { ThinkingLevel } from "@earendil-works/pi-agent-core";
+
+export const SUPPORTED_TOOL_NAMES = [
+  "read",
+  "bash",
+  "edit",
+  "write",
+  "grep",
+  "find",
+  "ls",
+] as const;
+
+export type SupportedToolName = (typeof SUPPORTED_TOOL_NAMES)[number];
+
+const supportedToolNames: ReadonlySet<string> = new Set(SUPPORTED_TOOL_NAMES);
+
+export function isSupportedToolName(value: unknown): value is SupportedToolName {
+  return typeof value === "string" && supportedToolNames.has(value);
+}
+
+export type WorkerSourceKind = "package" | "user" | "project";
+
+export interface WorkerSource {
+  readonly kind: WorkerSourceKind;
+  readonly filePath: string;
+}
+
+export interface WorkerModel {
+  readonly provider: string;
+  readonly modelId: string;
+}
+
+export interface WorkerCompaction {
+  readonly enabled?: boolean;
+  readonly reserveTokens?: number;
+  readonly keepRecentTokens?: number;
+}
+
+export type WorkerLifecycle = "one-shot" | "interactive";
+
+export interface WorkerDefinition {
+  readonly name: string;
+  readonly source: WorkerSource;
+  readonly description: string;
+  readonly systemPrompt: string;
+  readonly lifecycle: WorkerLifecycle;
+  readonly tools: readonly SupportedToolName[];
+  readonly skills?: readonly string[];
+  readonly model?: WorkerModel;
+  readonly thinking?: ThinkingLevel;
+  readonly compaction?: WorkerCompaction;
+}
+
+export type CatalogDiagnosticSeverity = "warning" | "error";
+
+export interface CatalogDiagnostic {
+  readonly severity: CatalogDiagnosticSeverity;
+  readonly source: WorkerSourceKind;
+  readonly message: string;
+  readonly filePath?: string;
+}
+
+export interface WorkerCatalog {
+  readonly workers: readonly WorkerDefinition[];
+  readonly diagnostics: readonly CatalogDiagnostic[];
+}
+
+export function createWorkerCatalog(
+  workers: readonly WorkerDefinition[],
+  diagnostics: readonly CatalogDiagnostic[] = [],
+): WorkerCatalog {
+  return {
+    workers: [...workers].sort(compareWorkersByName),
+    diagnostics: [...diagnostics],
+  };
+}
+
+export function findWorkerByName(
+  catalog: WorkerCatalog,
+  name: string,
+): WorkerDefinition | undefined {
+  return catalog.workers.find((worker) => worker.name === name);
+}
+
+function compareWorkersByName(left: WorkerDefinition, right: WorkerDefinition): number {
+  if (left.name < right.name) return -1;
+  if (left.name > right.name) return 1;
+  return 0;
+}

package/extension/{catalog.ts → catalog/discovery.ts}

@@ -5,19 +5,19 @@ import {
 } from "@earendil-works/pi-coding-agent";
 import { lstatSync, readdirSync, readFileSync } from "node:fs";
 import { basename, extname, join } from "node:path";
-import { fileURLToPath } from "node:url";
 import { Effect, Result, Schema, SchemaGetter, type SchemaIssue } from "effect";
 import type {
   CatalogDiagnostic,
   WorkerCatalog,
   WorkerDefinition,
   WorkerSourceKind,
-} from "./domain.js";
+} from "./definition.ts";
 import {
   createWorkerCatalog,
   isSupportedToolName,
   SUPPORTED_TOOL_NAMES,
-} from "./domain.js";
+} from "./definition.ts";
+import { PACKAGE_ROOT } from "../package-root.ts";
 
 const MAX_WORKER_BYTES = 64 * 1024;
 const THINKING_LEVELS = ["off", "minimal", "low", "medium", "high", "xhigh", "max"] as const;
@@ -302,7 +302,7 @@ function parseWorker(
 }
 
 function sourceDirectories(options: DiscoverWorkerCatalogOptions): CatalogSource[] {
-  const packageRoot = options.packageRoot ?? fileURLToPath(new URL("..", import.meta.url));
+  const packageRoot = options.packageRoot ?? PACKAGE_ROOT;
   const agentDir = options.agentDir ?? getAgentDir();
   const sources: CatalogSource[] = [
     { kind: "package", directory: join(packageRoot, "examples", "workers") },

package/extension/index.ts

@@ -6,9 +6,9 @@ import type {
 import {
   discoverWorkerCatalog,
   type DiscoverWorkerCatalogOptions,
-} from "./catalog.js";
-import { appendOrchestratorContract } from "./contract.js";
-import type { WorkerCatalog } from "./domain.js";
+} from "./catalog/discovery.ts";
+import type { WorkerCatalog } from "./catalog/definition.ts";
+import { applyOrchestratorContract } from "./parent/contract.ts";
 import {
   attachProcessHost,
   createProcessHost,
@@ -16,21 +16,18 @@ import {
   detachProcessHost,
   type ProcessHost,
   type ProcessHostAttachment,
-} from "./host.js";
+} from "./parent/process-host.ts";
 import {
   createStatusController,
   registerOrchestrationPresentation,
   type StatusController,
-} from "./presentation.js";
+  type WorkerStateSource,
+} from "./pi/presentation.ts";
 import {
-  registerOrchestrationTools,
+  classifyParentDispatches,
   type DispatchDecision,
-} from "./tools.js";
-
-const DISPATCH_TOOL_NAMES: ReadonlySet<string> = new Set([
-  "orchestrate",
-  "interactive_send",
-]);
+} from "./parent/dispatch-policy.ts";
+import { registerOrchestrationTools } from "./pi/tools.ts";
 
 interface StoredDispatchDecision extends DispatchDecision {
   readonly ownerSessionId: string;
@@ -45,7 +42,7 @@ export interface OrchestrationExtensionDependencies {
   getHost?(): ProcessHost;
   destroyHost?(host: ProcessHost): Promise<void>;
   discoverCatalog?(options: DiscoverWorkerCatalogOptions): WorkerCatalog;
-  createStatusController?(runtime: ProcessHost["runtime"]): StatusController;
+  createStatusController?(workerState: WorkerStateSource): StatusController;
 }
 
 export function createOrchestrationExtension(
@@ -77,10 +74,10 @@ export function createOrchestrationExtension(
     pi.on("session_start", (_event, ctx) => {
       host ??= dependencies.getHost?.() ?? createProcessHost();
       statusController ??=
-        dependencies.createStatusController?.(host.runtime) ??
-        createStatusController(host.runtime);
+        dependencies.createStatusController?.(host.orchestration) ??
+        createStatusController(host.orchestration);
       registerOrchestrationTools(pi, {
-        runtime: host.runtime,
+        orchestration: host.orchestration,
         getCatalog: catalogFor,
         getDispatchDecision: (toolCallId) =>
           dispatchDecisions.get(toolCallId) ?? { mode: "inline" },
@@ -106,7 +103,7 @@ export function createOrchestrationExtension(
     pi.on("before_agent_start", (event, ctx) => {
       cachedCatalog = discoverCatalogFor(ctx);
       return {
-        systemPrompt: appendOrchestratorContract(
+        systemPrompt: applyOrchestratorContract(
           event.systemPrompt,
           cachedCatalog,
         ),
@@ -120,26 +117,10 @@ export function createOrchestrationExtension(
       );
       const ownerSessionId = activeBinding?.ownerSessionId;
       if (!ownerSessionId) return;
-      // Sole dispatches and homogeneous orchestrate waves detach; mixed tools stay with the
-      // current parent turn, while a wave shares one boundary for one later synthesis turn.
-      const isOrchestrateGroup =
-        toolCalls.length > 1 &&
-        toolCalls.every((toolCall) => toolCall.name === "orchestrate");
-      const synthesisGroup = isOrchestrateGroup
-        ? { id: `orchestrate:${toolCalls[0]?.id ?? "group"}`, size: toolCalls.length }
-        : undefined;
-
-      for (const toolCall of toolCalls) {
-        if (!DISPATCH_TOOL_NAMES.has(toolCall.name)) continue;
-        const mode = isOrchestrateGroup || toolCalls.length === 1
-          ? "async"
-          : "inline";
-        dispatchDecisions.set(toolCall.id, {
-          mode,
+      for (const dispatch of classifyParentDispatches(toolCalls)) {
+        dispatchDecisions.set(dispatch.toolCallId, {
+          ...dispatch.decision,
           ownerSessionId,
-          ...(toolCall.name === "orchestrate" && synthesisGroup
-            ? { synthesisGroup }
-            : {}),
         });
       }
     });
@@ -148,6 +129,8 @@ export function createOrchestrationExtension(
       const decision = dispatchDecisions.get(event.toolCallId);
       dispatchDecisions.delete(event.toolCallId);
       if (!event.isError || !decision?.synthesisGroup) return;
+      // Group size is counted before admission, so a failed member must still be
+      // accounted for as skipped or the final synthesis turn will never trigger.
       host?.delivery.skipSynthesisGroupMember(
         decision.ownerSessionId,
         decision.synthesisGroup.id,