@zachwill/pi-orchestrate 0.1.0 → 0.2.0

package/README.md

@@ -1,6 +1,6 @@
 # Pi Orchestrate
 
-[`@zachwill/pi-orchestrate`](https://www.npmjs.com/package/@zachwill/pi-orchestrate) adds concurrent worker orchestration to [Pi](https://pi.dev). It helps a parent agent **delegate → dispatch → deliver**: delegate bounded work, dispatch independent tasks as one wave, then deliver one aggregate result to the parent.
+[`@zachwill/pi-orchestrate`](https://www.npmjs.com/package/@zachwill/pi-orchestrate) adds concurrent worker orchestration to [Pi](https://pi.dev). It helps a parent agent delegate bounded work, message independent workers concurrently, and synthesize each response as it arrives.
 
 ## Install
 
@@ -22,11 +22,13 @@ Pi Orchestrate adds exactly five tools:
 
 An `orchestrate` wave first performs atomic input, catalog, and model preflight. No worker starts if any task fails that preflight. After acceptance, worker resources start independently: a resource startup failure becomes that worker's `failed` result and does not roll back or stop its peers.
 
-All tasks in a wave execute concurrently, including a full 12-task wave. Pi Orchestrate applies no hidden concurrency throttle. The aggregate result preserves task order.
+All tasks in a wave execute concurrently, including a full 12-task wave. Pi Orchestrate applies no hidden concurrency throttle. Exact unchanged worker instructions remain visible in each tool call: collapsed calls preview every message, and expanded calls show every full brief. Titles are labels, never replacements for instructions.
 
 For asynchronous behavior, `orchestrate` or `worker_send` must be the sole tool call in its assistant message. Mixing either with any sibling tool call makes its wave inline and blocking. Inline work receives the parent turn's cancellation signal. Accepted async work does not retain that signal and continues independently.
 
-A successful async call yields the parent turn until aggregate results arrive. Do not poll `orchestration_status` for completion. If the owning session becomes inactive, its workers continue and completed results remain queued. Those results become available only when that exact owning session resumes; they are never delivered to another session.
+A successful async call yields the parent turn. Responses enter the transcript individually as workers settle, and the final response starts the parent's synthesis turn. The bottom widget is ephemeral and shows active work only; completed, failed, aborted, and reusable ready workers disappear immediately. Inline responses accumulate in the live tool output while the call blocks.
+
+`orchestration_status` is for diagnostics and recovery, never a normal completion mechanism. Do not poll it for completion. If the owning session becomes inactive, its workers continue and completed results remain queued. Those results become available only when that exact owning session resumes; they are never delivered to another session.
 
 Worker IDs identify live worker sessions. A reusable worker keeps the same worker ID across `worker_send` follow-ups, with each follow-up result belonging to a new wave. One-shot workers finish as `completed`. Reusable workers deliver as `ready` and wait for `worker_send` or `worker_close`. Use `worker_abort` only for active work, not to close a ready worker.
 
@@ -78,6 +80,7 @@ Frontmatter supports these fields:
 - `tools` and `lifecycle` are required. Grant the smallest useful tool set.
 - `model` is optional. When omitted, the worker inherits the parent model active when dispatched.
 - `thinking`, `skills`, and `compaction` are optional.
+- Omitted `skills` uses Pi's normal discovered skills. A nonempty `skills` list is an exact name allowlist, and `skills: []` disables skills.
 - `lifecycle` must be exactly `one-shot` or `reusable`.
 - The Markdown body must be nonempty.
 
@@ -91,4 +94,6 @@ Use one-shot workers for bounded investigation, review, and implementation. Use
 
 Worker sessions are isolated from the parent's conversational context, but they run in-process and are not sandboxes. They share the parent process's filesystem and environment permissions. Treat worker prompts, optional skills, models, and tool grants as trusted code.
 
+Workers use regular persisted Pi global settings, authentication, packages, extensions, skills, and context. Trusted projects also contribute their project settings and resources; untrusted projects do not. Extensions are active in print mode for the complete worker lifecycle, including resource discovery and provider request hooks. Pi Orchestrate excludes its own package before child extension factories execute, so workers remain direct children while other configured extensions—including provider integrations such as `@benvargas/pi-claude-code-use`—load normally. Worker definitions still provide the exact bounded tool allowlist.
+
 Pi Orchestrate performs no automatic filesystem writes. A worker writes only when its instructions and granted tools cause it to do so. Parallel workers must have non-overlapping write scopes, and the parent must inspect and verify their changes.

package/extension/catalog.ts

@@ -6,41 +6,79 @@ import {
 import { lstatSync, readdirSync, readFileSync } from "node:fs";
 import { basename, extname, join } from "node:path";
 import { fileURLToPath } from "node:url";
+import { Result, Schema, SchemaGetter, type SchemaIssue } from "effect";
 import type {
   CatalogDiagnostic,
-  SupportedToolName,
   WorkerCatalog,
   WorkerDefinition,
-  WorkerLifecycle,
   WorkerSourceKind,
 } from "./domain.js";
-import { isSupportedToolName } from "./domain.js";
+import { isSupportedToolName, SUPPORTED_TOOL_NAMES } from "./domain.js";
 
 const MAX_WORKER_BYTES = 64 * 1024;
-const KNOWN_FIELDS = new Set([
-  "name",
-  "description",
-  "model",
-  "thinking",
-  "tools",
-  "skills",
-  "compaction",
-  "lifecycle",
-]);
-const THINKING_LEVELS: ReadonlySet<string> = new Set([
-  "off",
-  "minimal",
-  "low",
-  "medium",
-  "high",
-  "xhigh",
-  "max",
-]);
-
-function isThinkingLevel(value: string): value is NonNullable<WorkerDefinition["thinking"]> {
-  return THINKING_LEVELS.has(value);
+const THINKING_LEVELS = ["off", "minimal", "low", "medium", "high", "xhigh", "max"] as const;
+
+const RequiredText = Schema.Trim.check(Schema.isNonEmpty());
+const StringListInput = Schema.Union([Schema.String, Schema.Array(Schema.String)]);
+
+function commaList<Item extends Schema.Constraint & { readonly Encoded: string }>(
+  item: Item,
+  allowEmpty = false,
+) {
+  const items = allowEmpty
+    ? Schema.Array(item)
+    : Schema.Array(item).check(Schema.isMinLength(1));
+  return StringListInput.pipe(
+    Schema.decodeTo(items, {
+      decode: SchemaGetter.transform((value) =>
+        typeof value === "string" ? value.split(",").map((entry) => entry.trim()) : value,
+      ),
+      encode: SchemaGetter.transform((value) => value),
+    }),
+  );
 }
 
+const ModelCoordinate = Schema.Trim.check(Schema.isPattern(/^[^/\s]+\/\S+$/)).pipe(
+  Schema.decodeTo(
+    Schema.Struct({ provider: Schema.NonEmptyString, modelId: Schema.NonEmptyString }),
+    {
+      decode: SchemaGetter.transform((coordinate) => {
+        const separator = coordinate.indexOf("/");
+        return {
+          provider: coordinate.slice(0, separator),
+          modelId: coordinate.slice(separator + 1),
+        };
+      }),
+      encode: SchemaGetter.transform(({ provider, modelId }) => `${provider}/${modelId}`),
+    },
+  ),
+);
+
+const NonNegativeInteger = Schema.Number.check(
+  Schema.isInt(),
+  Schema.isGreaterThanOrEqualTo(0),
+);
+const Compaction = Schema.Struct({
+  enabled: Schema.optionalKey(Schema.Boolean),
+  reserveTokens: Schema.optionalKey(NonNegativeInteger),
+  keepRecentTokens: Schema.optionalKey(NonNegativeInteger),
+});
+const ThinkingLevel = Schema.Trim.pipe(Schema.decodeTo(Schema.Literals(THINKING_LEVELS)));
+const WorkerFrontmatter = Schema.Struct({
+  name: RequiredText,
+  description: RequiredText,
+  model: Schema.optionalKey(ModelCoordinate),
+  thinking: Schema.optionalKey(ThinkingLevel),
+  tools: commaList(Schema.Literals(SUPPORTED_TOOL_NAMES)),
+  skills: Schema.optionalKey(commaList(Schema.NonEmptyString, true)),
+  compaction: Schema.optionalKey(Compaction),
+  lifecycle: Schema.Literals(["one-shot", "reusable"]),
+});
+const decodeWorkerFrontmatter = Schema.decodeUnknownResult(WorkerFrontmatter, {
+  errors: "all",
+  onExcessProperty: "error",
+});
+
 export interface CatalogFileStat {
   readonly size: number;
   isFile(): boolean;
@@ -65,18 +103,6 @@ interface CatalogSource {
   readonly directory: string;
 }
 
-interface WorkerFrontmatter {
-  readonly name?: unknown;
-  readonly description?: unknown;
-  readonly model?: unknown;
-  readonly thinking?: unknown;
-  readonly tools?: unknown;
-  readonly skills?: unknown;
-  readonly compaction?: unknown;
-  readonly lifecycle?: unknown;
-  readonly [field: string]: unknown;
-}
-
 const productionFileSystem: CatalogFileSystem = {
   readDirectory: (directory) => readdirSync(directory),
   inspect: (path) => lstatSync(path),
@@ -101,98 +127,128 @@ function isMissingPath(error: unknown): boolean {
   return typeof error === "object" && error !== null && "code" in error && error.code === "ENOENT";
 }
 
-function requiredString(value: unknown, field: string): string {
-  if (typeof value !== "string" || value.trim() === "") {
-    throw new Error(`frontmatter field '${field}' must be a non-empty string`);
-  }
-  return value.trim();
-}
-
-function optionalString(value: unknown, field: string): string | undefined {
-  if (value === undefined) return undefined;
-  return requiredString(value, field);
+interface UnexpectedPath {
+  readonly path: readonly PropertyKey[];
 }
 
-function stringList(value: unknown, field: string, required: boolean): string[] | undefined {
-  if (value === undefined) {
-    if (required) throw new Error(`frontmatter field '${field}' is required`);
-    return undefined;
-  }
-
-  const values = typeof value === "string" ? value.split(",").map((item) => item.trim()) : value;
-  if (!Array.isArray(values) || values.length === 0) {
-    throw new Error(`frontmatter field '${field}' must be a non-empty comma string or string array`);
-  }
-
-  const strings: string[] = [];
-  for (const item of values) {
-    if (typeof item !== "string" || item === "") {
-      throw new Error(`frontmatter field '${field}' must be a non-empty comma string or string array`);
-    }
-    strings.push(item);
+function collectUnexpectedPaths(
+  issue: SchemaIssue.Issue,
+  parentPath: readonly PropertyKey[] = [],
+): UnexpectedPath[] {
+  switch (issue._tag) {
+    case "Pointer":
+      return collectUnexpectedPaths(issue.issue, [...parentPath, ...issue.path]);
+    case "Composite":
+    case "AnyOf":
+      return issue.issues.flatMap((child) => collectUnexpectedPaths(child, parentPath));
+    case "Encoding":
+    case "Filter":
+      return collectUnexpectedPaths(issue.issue, parentPath);
+    case "UnexpectedKey":
+      return [{ path: parentPath }];
+    default:
+      return [];
   }
-  return strings;
 }
 
-function parseTools(value: unknown): SupportedToolName[] {
-  const tools: SupportedToolName[] = [];
-  for (const tool of stringList(value, "tools", true) ?? []) {
-    if (!isSupportedToolName(tool)) throw new Error(`unsupported tool '${tool}'`);
-    tools.push(tool);
-  }
-  return tools;
+function isUnknownRecord(value: unknown): value is Record<string, unknown> {
+  return typeof value === "object" && value !== null && !Array.isArray(value);
 }
 
-function parseOptionalBoolean(value: unknown, field: string): boolean | undefined {
-  if (value === undefined) return undefined;
-  if (typeof value !== "boolean") {
-    throw new Error(`frontmatter field '${field}' must be a boolean`);
-  }
-  return value;
+function fieldValue(frontmatter: unknown, field: string): unknown {
+  if (!isUnknownRecord(frontmatter)) return undefined;
+  return field in frontmatter ? frontmatter[field] : undefined;
 }
 
-function parseLifecycle(value: unknown): WorkerLifecycle {
-  if (value !== "one-shot" && value !== "reusable") {
-    throw new Error("frontmatter field 'lifecycle' must be 'one-shot' or 'reusable'");
-  }
-  return value;
+function listItems(value: unknown): readonly unknown[] {
+  if (typeof value === "string") return value.split(",").map((item) => item.trim());
+  return Array.isArray(value) ? value : [];
 }
 
-function parseCompaction(value: unknown): WorkerDefinition["compaction"] {
-  if (value === undefined) return undefined;
-  if (typeof value !== "object" || value === null || Array.isArray(value)) {
-    throw new Error("frontmatter field 'compaction' must be a mapping");
+function schemaDiagnostic(error: Schema.SchemaError, frontmatter: unknown): string {
+  const unexpected = collectUnexpectedPaths(error.issue);
+  const frontmatterFields = unexpected
+    .filter(({ path }) => path.length === 1 && typeof path[0] === "string")
+    .map(({ path }) => String(path[0]))
+    .sort(compareText);
+  if (frontmatterFields.length > 0) {
+    return `unknown frontmatter field${frontmatterFields.length === 1 ? "" : "s"}: ${frontmatterFields.join(", ")}`;
   }
 
-  const fields = Object.keys(value).sort(compareText);
-  const unknownFields = fields.filter(
-    (field) => field !== "enabled" && field !== "reserveTokens" && field !== "keepRecentTokens",
-  );
-  if (unknownFields.length > 0) {
-    throw new Error(
-      `unknown compaction field${unknownFields.length === 1 ? "" : "s"}: ${unknownFields.join(", ")}`,
-    );
+  const compactionFields = unexpected
+    .filter(
+      ({ path }) => path.length === 2 && path[0] === "compaction" && typeof path[1] === "string",
+    )
+    .map(({ path }) => String(path[1]))
+    .sort(compareText);
+
+  if (typeof frontmatter !== "object" || frontmatter === null || Array.isArray(frontmatter)) {
+    return "frontmatter must be a mapping";
   }
 
-  const enabledValue = "enabled" in value ? value.enabled : undefined;
-  const reserveTokens = "reserveTokens" in value ? value.reserveTokens : undefined;
-  const keepRecentTokens = "keepRecentTokens" in value ? value.keepRecentTokens : undefined;
-  const enabled = parseOptionalBoolean(enabledValue, "compaction.enabled");
+  const orderedFields = [
+    "name",
+    "description",
+    "model",
+    "thinking",
+    "tools",
+    "skills",
+    "compaction",
+    "lifecycle",
+  ];
+  const message = error.message;
+  const field = orderedFields.find((candidate) => message.includes(`["${candidate}"]`));
+  const value = field === undefined ? undefined : fieldValue(frontmatter, field);
 
-  if (
-    reserveTokens !== undefined &&
-    (typeof reserveTokens !== "number" || !Number.isSafeInteger(reserveTokens) || reserveTokens < 0)
-  ) {
-    throw new Error("frontmatter field 'compaction.reserveTokens' must be a non-negative integer");
+  if (field === "name" || field === "description") {
+    return `frontmatter field '${field}' must be a non-empty string`;
   }
-  if (
-    keepRecentTokens !== undefined &&
-    (typeof keepRecentTokens !== "number" || !Number.isSafeInteger(keepRecentTokens) || keepRecentTokens < 0)
-  ) {
-    throw new Error("frontmatter field 'compaction.keepRecentTokens' must be a non-negative integer");
+  if (field === "model") {
+    if (typeof value !== "string" || value.trim() === "") {
+      return "frontmatter field 'model' must be a non-empty string";
+    }
+    return "frontmatter field 'model' must use provider/model format";
   }
-
-  return { enabled, reserveTokens, keepRecentTokens };
+  if (field === "thinking") {
+    if (typeof value !== "string" || value.trim() === "") {
+      return "frontmatter field 'thinking' must be a non-empty string";
+    }
+    return `unsupported thinking level '${value.trim()}'`;
+  }
+  if (field === "tools") {
+    const items = listItems(value);
+    const validList = items.length > 0 && items.every(
+      (item) => typeof item === "string" && item !== "",
+    );
+    const unsupported = validList ? items.find(
+      (item) => typeof item === "string" && !isSupportedToolName(item),
+    ) : undefined;
+    if (typeof unsupported === "string") return `unsupported tool '${unsupported}'`;
+    if (value === undefined) return "frontmatter field 'tools' is required";
+    return "frontmatter field 'tools' must be a non-empty comma string or string array";
+  }
+  if (field === "skills") {
+    return "frontmatter field 'skills' must be a comma string or string array";
+  }
+  if (field === "compaction") {
+    if (compactionFields.length > 0) {
+      return `unknown compaction field${compactionFields.length === 1 ? "" : "s"}: ${compactionFields.join(", ")}`;
+    }
+    if (message.includes('["enabled"]')) {
+      return "frontmatter field 'compaction.enabled' must be a boolean";
+    }
+    if (message.includes('["reserveTokens"]')) {
+      return "frontmatter field 'compaction.reserveTokens' must be a non-negative integer";
+    }
+    if (message.includes('["keepRecentTokens"]')) {
+      return "frontmatter field 'compaction.keepRecentTokens' must be a non-negative integer";
+    }
+    return "frontmatter field 'compaction' must be a mapping";
+  }
+  if (field === "lifecycle") {
+    return "frontmatter field 'lifecycle' must be 'one-shot' or 'reusable'";
+  }
+  return "invalid worker definition";
 }
 
 function parseWorker(
@@ -200,65 +256,28 @@ function parseWorker(
   source: WorkerSourceKind,
   content: string,
 ): WorkerDefinition {
-  let parsed: ReturnType<typeof parseFrontmatter<WorkerFrontmatter>>;
+  let parsed: ReturnType<typeof parseFrontmatter>;
   try {
-    parsed = parseFrontmatter<WorkerFrontmatter>(content);
+    parsed = parseFrontmatter(content);
   } catch {
     throw new Error("frontmatter is not valid YAML");
   }
 
   const { frontmatter, body } = parsed;
-  if (typeof frontmatter !== "object" || frontmatter === null || Array.isArray(frontmatter)) {
-    throw new Error("frontmatter must be a mapping");
-  }
-
-  const unknownFields = Object.keys(frontmatter)
-    .filter((field) => !KNOWN_FIELDS.has(field))
-    .sort(compareText);
-  if (unknownFields.length > 0) {
-    throw new Error(
-      `unknown frontmatter field${unknownFields.length === 1 ? "" : "s"}: ${unknownFields.join(", ")}`,
-    );
+  const decoded = decodeWorkerFrontmatter(frontmatter);
+  if (Result.isFailure(decoded)) {
+    throw new Error(schemaDiagnostic(decoded.failure, frontmatter));
   }
 
-  const name = requiredString(frontmatter.name, "name");
+  const worker = decoded.success;
   const expectedName = basename(filePath, extname(filePath));
-  if (name !== expectedName) {
-    throw new Error(`frontmatter name '${name}' must match basename '${expectedName}'`);
+  if (worker.name !== expectedName) {
+    throw new Error(`frontmatter name '${worker.name}' must match basename '${expectedName}'`);
   }
-
-  const description = requiredString(frontmatter.description, "description");
-  const model = optionalString(frontmatter.model, "model");
-  if (model !== undefined && !/^[^/\s]+\/\S+$/.test(model)) {
-    throw new Error("frontmatter field 'model' must use provider/model format");
-  }
-
-  const thinking = optionalString(frontmatter.thinking, "thinking");
-  if (thinking !== undefined && !isThinkingLevel(thinking)) {
-    throw new Error(`unsupported thinking level '${thinking}'`);
-  }
-
-  const tools = parseTools(frontmatter.tools);
-  const skills = stringList(frontmatter.skills, "skills", false);
-  const compaction = parseCompaction(frontmatter.compaction);
-  const lifecycle = parseLifecycle(frontmatter.lifecycle);
   if (body.trim() === "") throw new Error("worker prompt body must not be empty");
 
   return {
-    name,
-    description,
-    model:
-      model === undefined
-        ? undefined
-        : {
-            provider: model.slice(0, model.indexOf("/")),
-            modelId: model.slice(model.indexOf("/") + 1),
-          },
-    thinking,
-    tools,
-    skills: skills ?? [],
-    compaction,
-    lifecycle,
+    ...worker,
     systemPrompt: body,
     source: { kind: source, filePath },
   };

package/extension/contract.ts

@@ -34,7 +34,9 @@ You are the parent orchestrator and own the task end to end.
 - Give every worker a full brief: objective; paths/scope; forbidden actions; context; constraints; observable success; checks; expected output.
 - Input, catalog, and model preflight is atomic before any worker starts. Resource startup failures become per-worker failed results and do not roll back peers.
 - For asynchronous behavior, \`orchestrate\` or \`worker_send\` must be the sole tool call in its assistant message. Mixing either with any sibling tool call makes its wave inline and blocking.
-- After an accepted async wave, yield the parent turn until aggregate results arrive. Do not duplicate delegated work or poll \`orchestration_status\` for completion.
+- Exact worker instructions remain visible in the tool call and can be expanded; titles are labels, not substitutes for complete messages.
+- After an accepted async wave, yield the parent turn. Worker responses arrive individually as each worker settles, and the final response starts parent synthesis. Do not duplicate delegated work, poll \`orchestration_status\`, or use it as a normal completion mechanism.
+- The active-work widget shows only workers currently starting, running, or stopping. Inline worker responses appear progressively in live tool output.
 - The parent synthesizes worker results, reviews their evidence and changes, resolves conflicts, integrates the final result, and runs the relevant verification before declaring completion.
 - Prefer one-shot workers. Use \`worker_send\` for follow-up work on a ready reusable worker, \`worker_close\` when that ready worker is finished, and \`worker_abort\` only when active work must stop.
 - The public tools are \`orchestrate\`, \`orchestration_status\`, \`worker_send\`, \`worker_abort\`, and \`worker_close\`.