@pify/swarm 0.4.0 → 0.6.0

package/README.md

@@ -1,16 +1,50 @@
 # @pify/swarm
 
-Coordinate multiple [pi](https://github.com/earendil-works/pi) agents working in parallel. One tool call fans a list of task items out to child agents — with auto-routing, a concurrency queue, a live widget, and one aggregated report.
+Run many [pi](https://github.com/earendil-works/pi) agents in parallel. One tool call fans a list of independent items out to child agents — with per-item routing, a concurrency queue, a live widget, and one aggregated report.
 
 Part of the [Pify suite](https://github.com/pifydev). Install with [`pify install swarm`](https://github.com/pifydev/cli) or `pi install npm:@pify/swarm`.
 
-## What it does
+## Why
 
-- **`swarm_run`** — fan out 1–12 independent items to parallel children (4 at a time, rest queued). Blocking by default: returns `N done, M error` plus a per-item report. `background: true` returns a `runId` immediately.
-- **`swarm_status`** — live per-item progress (`1:scout=running(3t) · 2:reviewer=queued`), full report once finished; completed runs survive `/reload`.
-- **Per-item auto-routing**: agent definitions can declare `match_patterns` (globs matched against path-like tokens in the item — longest wins) and `match_keywords`. `review src/auth.rs` routes to your Rust auditor; `test the login flow` to your tester; everything else falls back to the read-only `scout` — the fallback never mutates. Force one type for all items with `agent`.
-- **Shared context**: the `context` param is prepended to every item, so common constraints are written once.
-- **One agent catalog**: reads the same `.pi/agents/*.md` definitions as [`@pify/subagent`](https://github.com/pifydev/subagent) (description/tools/model/thinking/max_turns), plus the two routing keys:
+Some work is a list of things that do not depend on each other: audit twelve modules, summarise nine files, check every package for the same problem. Doing that in one conversation is slow and fills the context with material the main thread does not need. Doing it with twelve separate delegation calls is the same work typed twelve times.
+
+The catch is that "independent" is usually a small lie — the items do not depend on each other's *results*, but they may touch the same files. That is what the mailbox and worktree isolation below are for.
+
+## Tools
+
+### `swarm_run`
+
+| Parameter | Type | Notes |
+|---|---|---|
+| `items` | string[], 1–12 | One task per item; four run at a time, the rest queue |
+| `context` | string, optional | Prepended to every item, so shared constraints are written once |
+| `agent` | string, optional | Force one agent type for all items instead of routing |
+| `isolation` | `"worktree"`, optional | Give each item its own git worktree — use it when items write |
+| `mailbox` | boolean, optional | Give the children `swarm_post` / `swarm_inbox` |
+| `background` | boolean, optional | Return a `runId` immediately instead of blocking |
+
+Blocking by default: returns `N done, M error` plus a per-item report.
+
+### `swarm_status`
+
+| Parameter | Type | Notes |
+|---|---|---|
+| `runId` | string, optional | Defaults to the most recent run |
+
+Live per-item progress (`1:scout=running(3t) · 2:reviewer=queued`), and the full report once the run finishes. Completed runs survive `/reload`.
+
+### `swarm_post` / `swarm_inbox`
+
+Registered for the children only, and only when `mailbox: true`.
+
+- `swarm_post(message)` — tell the siblings something that changes their work: a shared file you modified, a convention you had to pick, a blocker they will hit too.
+- `swarm_inbox()` — read what the others have posted since your last check.
+
+Without it, parallel agents cannot see each other, so two of them cheerfully fix the same shared helper in two different ways. It is deliberately not a chat: no addressing, no waiting, no replies. An append-only log per run, and an agent never sees its own posts echoed back. A torn line from two simultaneous appends is skipped rather than failing the read.
+
+## Per-item routing
+
+Agent definitions declare what they are for, and each item picks its own:
 
 ```markdown
 ---
@@ -21,23 +55,28 @@ match_keywords: rust, memory safety
 ---
 ```
 
-- **Independence by design**: items share nothing, children cannot spawn children, and each child is capped at its agent's `max_turns`.
+`match_patterns` are globs matched against path-like tokens in the item — the longest match wins, so a specific rule beats a general one. `match_keywords` match the item's words. `review src/auth.rs` routes to the Rust auditor; `test the login flow` to a tester; anything matching nothing falls back to the read-only `scout`, so **the fallback can never mutate**.
 
-## Where this sits in the suite
+The catalog is the same `.pi/agents/*.md` one [`@pify/subagent`](https://github.com/pifydev/subagent) reads — `description`, `tools`, `model`, `thinking`, `max_turns` — plus the two routing keys. Project-local definitions load only once pi's project trust has been granted.
 
-`@pify/subagent` = one child, one task. `@pify/swarm` = many independent items at once. `@pify/workflow` = deterministic scripted orchestration. Pick the smallest one that fits.
+## Behaviour
 
-## Mailbox (v0.3)
+- **Independence by design.** Items share nothing, children cannot spawn children, and each child is capped at its agent's `max_turns`.
+- **Stopping stops the children.** Pressing Esc, or switching away from the session, aborts every live child rather than leaving them talking to the provider on your money. A cancelled run keeps that verdict — it is never reported as done — and `swarm_status` shows what the items that did finish produced.
+- **Isolated runs clean up after themselves.** With `isolation: "worktree"`, a worktree whose child changed nothing is removed along with its branch; otherwise a read-only step left one of each behind on every run. Anything uncommitted, and any commit the child made, is kept and reported.
 
-`swarm_run(items, { mailbox: true })` gives every agent two extra tools:
+## A background run comes back to you
 
-- `swarm_post(message)` — tell the siblings something that changes their work: a shared file you modified, a convention you had to pick, a blocker they will hit too.
-- `swarm_inbox()` — read what the others posted since your last check.
+`swarm_status` on a run still in flight used to say "still running", which left the model one option: ask again. Now the aggregated report is **delivered** into the conversation when the run finishes, and asking early returns a structured result carrying `retryable`, the elapsed time and `pollRequired: false` — a normal answer rather than an error, because a tool error over a condition only time resolves invites the model's retry machinery into a loop.
 
-Without it, parallel agents cannot see each other, so two of them cheerfully fix the same shared helper in two different ways. It is deliberately not a chat: no addressing, no waiting, no replies — an append-only log per run, and an agent never sees its own posts echoed back. A torn line from two simultaneous appends is skipped rather than failing the read.
+## Command
+
+`/swarm` — runs in this session, and the agent types available for routing.
+
+## Where this sits in the suite
+
+[`@pify/subagent`](https://github.com/pifydev/subagent) is one child and one task. `@pify/swarm` is many independent items at once. [`@pify/workflow`](https://github.com/pifydev/workflow) is deterministic scripted orchestration for when the steps genuinely depend on each other. Pick the smallest one that fits.
 
 ## License
 
 MIT © [Pify maintainers](https://github.com/pifydev)
-
-**Isolated runs clean up after themselves** (v0.4): a worktree whose child changed nothing is removed along with its branch — otherwise a read-only step left one of each behind, per run. Anything uncommitted, or any commit the child made, is kept and reported.

package/extensions/swarm.ts

@@ -25,6 +25,8 @@ import { Text } from "@earendil-works/pi-tui";
 import { Type } from "typebox";
 
 import { BUILTIN_AGENTS } from "../src/builtin.ts";
+import { LiveChildren, cancelNote, type CancelReason } from "../src/cancel.ts";
+import { DELIVERY_TYPE, deliveryMessage, pendingResult } from "../src/pending.ts";
 import { createIsolationWorktree, isolationNote, removeIfUnchanged } from "../src/isolate.ts";
 import { formatInbox, mailboxDir, mailboxPrompt, postMessage, readInbox } from "../src/mailbox.ts";
 import { parseAgentFile } from "../src/frontmatter.ts";
@@ -77,6 +79,8 @@ function loadDefs(cwd: string, agentDir: string): Map<string, AgentDef> {
 export default function swarm(pi: ExtensionAPI) {
   let defs = new Map<string, AgentDef>();
   const runs = new Map<string, SwarmRun>();
+  /** Live child sessions per run, so a stop actually reaches the children. */
+  const live = new LiveChildren();
   let activeRun: SwarmRun | null = null;
   let runCounter = 0;
   let lastUiCtx: UiContext | null = null;
@@ -86,7 +90,7 @@ export default function swarm(pi: ExtensionAPI) {
     lastUiCtx = ctx;
     const run = activeRun;
     const now = Date.now();
-    if (!run || (run.status === "done" && (run.finishedAt ?? 0) < now - 15_000)) {
+    if (!run || (run.status !== "running" && (run.finishedAt ?? 0) < now - 15_000)) {
       ctx.ui.setWidget("swarm", undefined);
       return;
     }
@@ -151,6 +155,7 @@ export default function swarm(pi: ExtensionAPI) {
 
   async function runItem(
     ctx: UiContext,
+    runId: string,
     def: AgentDef,
     item: ItemState,
     context: string,
@@ -161,6 +166,7 @@ export default function swarm(pi: ExtensionAPI) {
     renderWidget();
     let session: AgentSession | null = null;
     let unsubscribe: (() => void) | null = null;
+    let releaseLive: (() => void) | null = null;
     try {
       let model = ctx.model ?? null;
       if (def.model) {
@@ -198,6 +204,7 @@ export default function swarm(pi: ExtensionAPI) {
         }),
       });
       session = created.session;
+      releaseLive = live.register(runId, session);
 
       unsubscribe = session.subscribe((event) => {
         if (event.type === "message_end" && (event as { message?: { role?: string } }).message?.role === "assistant") {
@@ -232,6 +239,7 @@ export default function swarm(pi: ExtensionAPI) {
       item.status = "error";
       item.error = err instanceof Error ? err.message : String(err);
     } finally {
+      if (releaseLive) releaseLive();
       if (unsubscribe) {
         try {
           unsubscribe();
@@ -264,6 +272,9 @@ export default function swarm(pi: ExtensionAPI) {
     const queue = [...run.items];
     const workers = Array.from({ length: Math.min(DEFAULT_CONCURRENCY, queue.length) }, async () => {
       for (;;) {
+        // A cancelled run stops taking new items; the ones already in flight
+        // were aborted by cancelRun.
+        if (run.status === "cancelled") return;
         const item = queue.shift();
         if (!item) return;
         const def = routeItem(item.item, defs, fixed);
@@ -271,24 +282,43 @@ export default function swarm(pi: ExtensionAPI) {
         if (isolate) {
           try {
             const iso = createIsolationWorktree(ctx.cwd, run.runId + "-i" + (item.index + 1));
-            await runItem(ctx, def, item, context, iso.path, mailbox);
+            await runItem(ctx, run.runId, def, item, context, iso.path, mailbox);
             if (item.result !== null) item.result = `${item.result}\n\n${isolationNote(iso)}`;
           } catch (err) {
             item.status = "error";
             item.error = err instanceof Error ? err.message : String(err);
           }
         } else {
-          await runItem(ctx, def, item, context, undefined, mailbox);
+          await runItem(ctx, run.runId, def, item, context, undefined, mailbox);
         }
       }
     });
     await Promise.all(workers);
-    run.status = "done";
+    if (run.status !== "cancelled") run.status = "done";
     run.finishedAt = Date.now();
     pi.appendEntry(RUN_ENTRY, run);
     renderWidget();
   }
 
+  /**
+   * Stop a run and every child it started. Both meanings of "stop" — the
+   * user's abort and session teardown — come through here.
+   */
+  function cancelRun(run: SwarmRun, reason: CancelReason): void {
+    const stopped = live.abortRun(run.runId);
+    if (run.status === "running") {
+      run.status = "cancelled";
+      run.finishedAt = Date.now();
+    }
+    for (const item of run.items) {
+      if (item.status === "running" || item.status === "queued") {
+        item.status = "aborted";
+        item.error = cancelNote(reason, stopped);
+      }
+    }
+    renderWidget();
+  }
+
   // ── Tools ────────────────────────────────────────────────────────────
 
   pi.registerTool({
@@ -323,7 +353,7 @@ export default function swarm(pi: ExtensionAPI) {
         isolation?: string;
         mailbox?: boolean;
       },
-      _signal,
+      signal,
       _onUpdate,
       ctx,
     ) {
@@ -359,9 +389,36 @@ export default function swarm(pi: ExtensionAPI) {
       activeRun = run;
       renderWidget(uiCtx);
 
+      // Esc must reach the children. A background run outlives this tool call
+      // by design, so its signal is not its cancel button.
+      let stopListening: (() => void) | null = null;
+      if (signal && !run.background) {
+        const onAbort = () => cancelRun(run, "user-abort");
+        if (signal.aborted) onAbort();
+        else {
+          signal.addEventListener("abort", onAbort, { once: true });
+          stopListening = () => signal.removeEventListener("abort", onAbort);
+        }
+      }
+
       if (run.background) {
         void executeRun(uiCtx, run, params.context ?? "", params.agent, params.isolation === "worktree", params.mailbox === true).then(() => {
-          notify(uiCtx, `swarm ${run.runId} finished — collect with swarm_status`, "info");
+          notify(uiCtx, `swarm ${run.runId} finished`, "info");
+          // The report goes to the agent, not only to the screen — otherwise
+          // asking again was its only way to find out.
+          try {
+            pi.sendMessage(
+              {
+                customType: DELIVERY_TYPE,
+                content: deliveryMessage(run.runId, "swarm", buildReport(run)),
+                display: true,
+                details: { runId: run.runId, status: run.status, items: run.items.length },
+              },
+              { deliverAs: "followUp", triggerTurn: true },
+            );
+          } catch {
+            // Delivery is a convenience; swarm_status still works.
+          }
         });
         return {
           content: [
@@ -371,7 +428,11 @@ export default function swarm(pi: ExtensionAPI) {
         };
       }
 
-      await executeRun(uiCtx, run, params.context ?? "", params.agent, params.isolation === "worktree", params.mailbox === true);
+      try {
+        await executeRun(uiCtx, run, params.context ?? "", params.agent, params.isolation === "worktree", params.mailbox === true);
+      } finally {
+        if (stopListening) stopListening();
+      }
       return {
         content: [{ type: "text", text: buildReport(run) }],
         details: { runId: run.runId },
@@ -389,7 +450,23 @@ export default function swarm(pi: ExtensionAPI) {
     async execute(_id, params: { runId?: string }) {
       const run = params.runId ? runs.get(params.runId.trim()) : activeRun ?? [...runs.values()].pop();
       if (!run) throw new Error("No swarm runs this session.");
-      const text = run.status === "done" ? buildReport(run) : buildStatusLine(run);
+      if (run.status === "running") {
+        const pending = pendingResult({
+          id: run.runId,
+          kind: "running",
+          startedAt: run.startedAt,
+          now: Date.now(),
+          collectWith: "swarm_status",
+        });
+        return { content: [{ type: "text", text: pending.text }], details: pending.details as never };
+      }
+      const text =
+        run.status === "cancelled"
+          ? "This run was cancelled before it finished. Below is what the items that did complete produced.\n" +
+            buildReport(run)
+          : run.status === "done"
+            ? buildReport(run)
+            : buildStatusLine(run);
       return { content: [{ type: "text", text }], details: { runId: run.runId, status: run.status } };
     },
   });
@@ -404,7 +481,7 @@ export default function swarm(pi: ExtensionAPI) {
       const e = entry as { type?: string; customType?: string; data?: unknown };
       if (e.type !== "custom" || e.customType !== RUN_ENTRY || !isRecord(e.data)) continue;
       const run = e.data as unknown as SwarmRun;
-      if (typeof run.runId === "string" && run.status === "done") {
+      if (typeof run.runId === "string" && run.status !== "running") {
         runs.set(run.runId, run);
         const n = Number.parseInt(run.runId.slice(1), 10);
         if (Number.isFinite(n) && n > runCounter) runCounter = n;
@@ -414,6 +491,10 @@ export default function swarm(pi: ExtensionAPI) {
   });
 
   pi.on("session_shutdown", async (_event, ctx) => {
+    // A run cannot outlive the session that owns it.
+    for (const run of runs.values()) {
+      if (run.status === "running") cancelRun(run, "session-switch");
+    }
     if (ctx.hasUI) ctx.ui.setWidget("swarm", undefined);
   });
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pify/swarm",
-  "version": "0.4.0",
+  "version": "0.6.0",
   "description": "Coordinate multiple pi agents in parallel: swarm_run fan-out with per-item auto-routing, concurrency queue, aggregated reports",
   "keywords": [
     "pi-package",

package/src/cancel.ts

@@ -0,0 +1,104 @@
+/**
+ * Stopping means stopping the children too.
+ *
+ * A run in this package is not one process: it is a tree of child agent
+ * sessions, each with its own provider connection. The tool that started them
+ * is handed an AbortSignal and the extension is told when the session goes
+ * away — and until now neither reached the children. Pressing Esc, or
+ * switching sessions with a run in flight, marked a record "aborted" while the
+ * children kept talking to the provider on the user's money, writing into a
+ * conversation nobody was reading.
+ *
+ * So every live child registers here, and the two places that mean "stop"
+ * abort all of them. (The rule is FradSer-adjacent prior art: @zhushanwen's
+ * subagent-workflow terminates running runs on session switch or shutdown
+ * rather than letting them outlive the session that owns them.)
+ */
+
+/** The part of a child agent session this module needs. */
+export interface Abortable {
+  abort(): unknown;
+}
+
+export type CancelReason = "user-abort" | "session-switch" | "timeout";
+
+/**
+ * Live child sessions, grouped by the run that owns them. Registration
+ * returns its own release, so a child that finishes normally leaves no trace
+ * and cannot be aborted twice.
+ */
+export class LiveChildren {
+  private byRun = new Map<string, Set<Abortable>>();
+
+  register(runId: string, child: Abortable): () => void {
+    let set = this.byRun.get(runId);
+    if (!set) {
+      set = new Set();
+      this.byRun.set(runId, set);
+    }
+    set.add(child);
+    return () => {
+      const current = this.byRun.get(runId);
+      if (!current) return;
+      current.delete(child);
+      if (current.size === 0) this.byRun.delete(runId);
+    };
+  }
+
+  /** How many children of this run are still live. */
+  count(runId: string): number {
+    return this.byRun.get(runId)?.size ?? 0;
+  }
+
+  /** Total live children across every run. */
+  total(): number {
+    let sum = 0;
+    for (const set of this.byRun.values()) sum += set.size;
+    return sum;
+  }
+
+  /**
+   * Abort every live child of one run and return how many were stopped. A
+   * child that throws from abort() is still counted and still dropped: the
+   * point is that nothing is left holding a connection, and one stubborn
+   * child must not spare the others.
+   */
+  abortRun(runId: string): number {
+    const set = this.byRun.get(runId);
+    if (!set) return 0;
+    let stopped = 0;
+    for (const child of [...set]) {
+      try {
+        const result = child.abort();
+        // abort() is async in pi; a rejection here is not ours to surface.
+        void Promise.resolve(result).catch(() => {});
+      } catch {
+        // already gone
+      }
+      stopped++;
+    }
+    this.byRun.delete(runId);
+    return stopped;
+  }
+
+  /** Abort every live child of every run. */
+  abortAll(): number {
+    let stopped = 0;
+    for (const runId of [...this.byRun.keys()]) stopped += this.abortRun(runId);
+    return stopped;
+  }
+}
+
+/** One line for the run log, naming who stopped it and what that cost. */
+export function cancelNote(reason: CancelReason, stopped: number): string {
+  const children =
+    stopped === 0 ? "no child agents were running" : `${stopped} child agent${stopped === 1 ? "" : "s"} stopped`;
+  switch (reason) {
+    case "user-abort":
+      return `Cancelled by the user — ${children}. Work already finished is kept; the run itself did not complete.`;
+    case "session-switch":
+      return `The session went away, so the run was terminated — ${children}. Tokens already spent are not recoverable; start a new run for a result.`;
+    case "timeout":
+      return `The run exceeded its time limit — ${children}.`;
+  }
+}

package/src/pending.ts

@@ -0,0 +1,86 @@
+/**
+ * Telling an agent to wait, without teaching it to poll.
+ *
+ * A background run gave the model exactly one way to find out it had
+ * finished: call the status tool again. "Still running — call agent_result
+ * later" is an instruction to spin, and models follow it, burning a turn and a
+ * request per check while the thing they are waiting for has not moved.
+ *
+ * Two halves fix that, and only together:
+ *
+ *   - a not-ready answer that is a normal structured result rather than an
+ *     error, carrying `retryable` and saying what to do *instead* of waiting.
+ *     Throwing would be worse than useless — a tool error invites the model's
+ *     own retry machinery into a loop over a condition that time, not
+ *     retrying, resolves;
+ *   - a push when the run actually finishes, so waiting is never the only
+ *     option on the table.
+ *
+ * Pure: the shapes and the words. The extension owns the clock and the host.
+ */
+
+export type PendingKind = "queued" | "running";
+
+export interface PendingInput {
+  /** The id the caller would poll with. */
+  id: string;
+  kind: PendingKind;
+  startedAt: number;
+  now: number;
+  /** What the caller asks for to collect it, e.g. `agent_result`. */
+  collectWith: string;
+}
+
+export interface PendingResult {
+  text: string;
+  details: {
+    id: string;
+    status: PendingKind;
+    /** True: this will resolve on its own. It is a wait, not a failure. */
+    retryable: boolean;
+    elapsedMs: number;
+    /** False, and load-bearing: there is nothing to poll for. */
+    pollRequired: false;
+  };
+}
+
+function elapsed(ms: number): string {
+  if (ms < 1000) return "just started";
+  const seconds = Math.round(ms / 1000);
+  if (seconds < 60) return `${seconds}s so far`;
+  return `${Math.floor(seconds / 60)}m ${seconds % 60}s so far`;
+}
+
+/**
+ * The answer to "is it done yet". It says no, says why that is fine, and
+ * closes the loop the question came from.
+ */
+export function pendingResult(input: PendingInput): PendingResult {
+  const ms = Math.max(0, input.now - input.startedAt);
+  const state = input.kind === "queued" ? "queued behind the concurrency cap" : "still running";
+  return {
+    text: [
+      `${input.id} is ${state} (${elapsed(ms)}).`,
+      "",
+      "Do not poll for it. The result is delivered to you automatically the moment it lands,",
+      `so there is nothing to wait for here — carry on with other work, or finish your turn and`,
+      `you will be picked back up. ${input.collectWith} is only needed if you want it early.`,
+    ].join("\n"),
+    details: { id: input.id, status: input.kind, retryable: true, elapsedMs: ms, pollRequired: false },
+  };
+}
+
+/** How a finished run introduces itself when it arrives unasked. */
+export function deliveryMessage(id: string, label: string, body: string): string {
+  return [
+    `<${label}_result id="${id}">`,
+    body.trim(),
+    `</${label}_result>`,
+    "",
+    `This is ${id}, which you started in the background; it has just finished and this is its report.`,
+    "Fold it into what you are doing. If you had already moved on, say what it changes — or that it changes nothing.",
+  ].join("\n");
+}
+
+/** The custom-message type a delivered result travels under. */
+export const DELIVERY_TYPE = "pify-background-result";

package/src/types.ts

@@ -61,7 +61,11 @@ export interface ItemState {
   error: string | null;
 }
 
-export type RunStatus = "running" | "done";
+/**
+ * "cancelled" is its own outcome, not a completion: someone stopped the run,
+ * and calling it done would report results nobody produced.
+ */
+export type RunStatus = "running" | "done" | "cancelled";
 
 export interface SwarmRun {
   runId: string;