pi-agent-replicate 0.0.0-stage → 1.4.1

package/docs/development.md

@@ -0,0 +1,27 @@
+# Development
+
+```bash
+npm test
+npm run lint
+npm pack --dry-run
+npx tsc --noEmit index.ts
+node --check index.js
+git diff --check
+```
+
+Run selected isolated real-Pi and Herdr cases from inside Herdr:
+
+```bash
+npm run test:integration -- test/integration/replication.test.ts
+```
+
+For a manual TUI check with a deterministic provider, set `PI_INTEGRATION_TUI_DIR` to a fresh ignored directory and `PI_INTEGRATION_IDE_ENTRY` to the current IDE source entry. Run only this case with the intended Pi binary on `PATH`:
+
+```bash
+node --experimental-strip-types test/integration/run-isolated.ts \
+  --test-name-pattern='renders an active isolated replica' test/integration/replication.test.ts
+```
+
+The case reloads Pi and records the loaded IDE in `runtime.json`. The applied and failed cases each write to their own subdirectory. Use its `pane.json` to attach a read-only observer to the private Herdr endpoint. Capture the active pane, then create `release-child` in the inspection directory. Capture the delivered result, then create `finish` so the runner can clean up. Both observation gates time out after five minutes. Do not use the user's persistent Herdr server.
+
+See [RELEASING.md](../RELEASING.md) for release checks.

package/docs/public-subagent-tree-api.md

@@ -0,0 +1,58 @@
+# Public subagent tree API
+
+`pi-agent-replicate` exports a registration-free API for installed Pi extensions running in Herdr. Importing the package root does not register model-facing tools.
+
+## Create and launch
+
+The model-facing `replicate` tool uses source-snapshot worktrees and host-owned joins. This separate API still supports named roles, bare children and managed worktrees. Its ownership and runtime contracts are unchanged.
+
+```ts
+import {
+  createSubagentTree,
+  type SubagentTreeResult,
+} from "pi-agent-replicate";
+
+const tree = createSubagentTree({ pi, ctx, metadata: { requestId: "search-1" } });
+const child = await tree.launchChild({
+  parentId: tree.callerId,
+  name: "search-router",
+  task: "Find the requested records.",
+  tools: ["read"],
+});
+const childResult = await child.result;
+const result: SubagentTreeResult = await tree.result;
+```
+
+A newly created root caller launches with `parentId: tree.callerId`. An attached child branch launches its own descendants with `parentId: handle.ownerId`, where `handle` is that branch's own handle:
+
+```ts
+const handle = attachSubagentTree({ pi, ctx });
+const grandchild = await handle.launchChild({
+  parentId: handle.ownerId,
+  name: "detail-search",
+  task: "Search the narrower records.",
+  tools: ["read"],
+});
+```
+
+The package owns `callerId`, `treeId`, `ownerId`, node IDs, parent IDs, and the opaque `ownershipToken`. Consumer metadata is JSON-compatible and limited to 64 KiB of serialized UTF-8 bytes. Returned metadata, results, and the `children` map are read-only. Child result metadata carries those immutable identities plus the consumer value.
+
+Launching accepts explicit `cwd` and `worktree` options and reuses the managed-worktree contract described in [Worktree subagents](worktree-subagents.md). Results carry a `SubagentWorktree` with the created path, branch, base, and observed Git state. The package keeps every worktree workspace, including after failures; the caller owns review, integration, and cleanup.
+
+## Reattach and fork
+
+Internal Pi launches carry their child lifecycle binding in the session, so moving a child does not depend on copying a `.lineage.json` sidecar. This does not replace a consumer's explicit callback reattachment described below; automatic model-facing result delivery and consumer callback delivery are separate contracts.
+
+A consumer calls `attachSubagentTree` from every fresh `session_start` with the current `pi`, `ctx`, and caller credentials, then calls `bindFinalCallback()` on the returned handle. For `/reload`, `/new`, and `/resume`, this replaces the stale session references. For `/fork`, pass `forkLineage: { previousSessionFile }`. The API checks the previous binding and the new session header's `parentSession` before replacing the active context. A failed check leaves the old context untouched.
+
+The attached handle rebuilds its read-only `children` map. Each child keeps the same awaitable result identity, including answers that settled while the consumer was unbound. Only the original caller branch may call `bindFinalCallback`; callback bindings are versioned, and a successful callback is delivered once. A callback failure remains pending for a later binding.
+
+## Settlement and cancellation
+
+`tree.result` resolves once the root has a clean or explicit-empty settled result and every queued or launched descendant has settled. A root provider error, launch failure, or unexpected abort fails the tree. Descendant errors remain in bounded lifecycle rows. Intentional Escape is not a final tree result. Node rows report actual pane evidence (`open`) and bounded session and error facts; full descendant outputs remain on child promises.
+
+`cancel()` returns the one authoritative terminal record. If the tree already has a terminal record, `cancel()` returns that existing result and performs no further work. Otherwise it stops new reservations, waits for in-flight launches to settle, closes known descendant panes, and confirms both pane absence and captured process exit. A confirmed cancellation yields `state: "cancelled"`. A termination it cannot confirm — missing identity or surviving resources — yields `state: "failed"` with error code `cancel_termination_failed` and the retained recovery evidence. Rows inside a failed cancellation record carry conservative recovery flags, notably `open: true`; these are not verified current pane state. Late child records cannot replace a terminal record.
+
+## Public exports
+
+The root exports `createSubagentTree`, `attachSubagentTree`, `SubagentTreeHandle`, `SubagentChildHandle`, launch/result/metadata types, `ForkLineage`, and the error classes `SubagentTreeError`, `SubagentTreeValidationError`, and `SubagentTreeOwnershipError`.

package/docs/retained-roles.md

@@ -0,0 +1,71 @@
+# Retained role launcher
+
+This reference applies to the package-root API and retained internal launcher, not the model-facing `replicate` tool.
+
+## Retained runtime policy
+
+Model and thinking are resolved independently:
+
+1. Use the value from the selected role's frontmatter.
+2. If the role omits it, inherit the value from the parent session.
+
+This role policy applies only to the retained internal launcher and package-root API. `replicate` uses the parent runtime without resolving a role. Bare spawns remain supported by those retained paths. They inherit both values. Role models must be one exact authenticated `provider/model-id`. Model fallback lists and `config.json` model routing are not supported.
+
+## Custom roles for the retained API
+
+The package has no built-in roles. Add roles in one of these places:
+
+| Source | Path |
+| --- | --- |
+| Project | `.pi/agents/<name>.md` |
+| Global | `$PI_CODING_AGENT_DIR/agents/<name>.md` (normally `~/.pi/agent/agents/<name>.md`) |
+| Role pack | An installed Pi package registered through the role discovery event |
+
+Priority is project, then global, then role pack. Duplicate role-pack names are rejected.
+
+Example:
+
+```markdown
+---
+name: security-reviewer
+description: Reviews a bounded change for security problems
+model: provider/exact-model-id
+thinking: high
+tools: read, bash
+spawning: false
+auto-exit: true
+---
+
+Review only the requested change. Report concrete findings with file references.
+```
+
+Useful frontmatter fields:
+
+- `name`, `description`
+- `model`, `thinking`
+- `tools`, `skills`, `deny-tools`
+- `spawning`, `allow-self-spawn`
+- `auto-exit`, `interactive`
+- `session-mode`: `standalone`, `lineage-only`, or `fork`
+- `cwd`
+- `system-prompt`: `append` or `replace`
+- `disable-model-invocation`
+
+Requested skills are saved in the child's conversation once. Reloading or resuming that conversation reuses the saved skills instead of adding another initialization message.
+
+Definitions that request an external `cli` fail before Herdr creates resources.
+
+### Role packs
+
+A Pi package can register one absolute Markdown file or a directory of direct `.md` children through this event:
+
+```typescript
+pi.events.on(
+  "pi-herdr-subagents:roles:discover:v1",
+  (request: { apiVersion: number; register(path: string): void }) => {
+    if (request.apiVersion === 1) request.register(absoluteRolesDirectory);
+  },
+);
+```
+
+A role-pack extension must remove its listener on `session_shutdown`.

package/docs/settled-lifecycle-contract.md

@@ -0,0 +1,71 @@
+# Settled lifecycle contract
+
+This document defines how child turns become one parent result. The TypeScript source for settled outcome shapes is `pi-extension/subagents/settled-contract.ts`.
+
+## Settled turns stay local
+
+A settled assistant entry is still classified for child control:
+
+- `clean`: non-empty assistant text with no final provider error.
+- `empty`: a settled assistant entry with no usable text.
+- `error`: `stopReason: "error"`.
+- `intentional-abort`: `stopReason: "aborted"` after the parent recorded an interrupt request.
+- `unexpected-abort`: `stopReason: "aborted"` without an interrupt request.
+
+These boundaries do not publish parent results. A persistent child can settle many times while it remains open. Its earlier answers, empty turns, errors, and aborts stay in its own session.
+
+For a replica, a clean nonempty response can record terminal intent only after the last descendant result is processed. Earlier responses stay local. A terminal provider error, Escape, an empty required summary or an unsaved close records failure, saves completion evidence and closes after descendant drain. That failure remains the terminal outcome across reload and later turns that process pending work, even if a later turn succeeds. User input does not suppress replica completion. A saved final response without settlement evidence is preserved as history, never promoted to success. Its original receives error facts and a file of all post-fork textual assistant answers, excluding inherited answers, tooling and hidden reasoning.
+
+If history export fails, the terminal result reports that error instead of retrying completion forever. An unknown inherited-answer boundary never authorizes exporting the whole conversation. If the session file cannot be checkpointed, terminal facts and source checkpointing still proceed. The checkout stays available, integration is blocked without advertising a saved-copy continuation.
+
+A queued self-reload is unfinished work. The acknowledgement before reload is not a terminal answer, and the stopped-replica timer stays suspended until reload continues the session or reports that it was blocked. An old runtime must not publish completion after shutdown, even if its descendant wait finishes later.
+
+A stopped model does not imply finished session work. Replicas wait for active tools, background terminal completion messages, pending background request receipts, reported live Linux descendant PIDs and their observed children, and their own replicas. The copy processes completion messages before returning. New work invalidates an older shutdown decision, including one already waiting for descendants. The idle safety timer cannot bypass this guard. Terminal completion remains authoritative for its exit status; observing the same PID exit must not replace it with a status-less duplicate. Work receipts and process ownership are read from the finalized `tool_execution_end` result, after all `tool_result` transformations, including nested calls. A successful terminal deletion retires only that terminal's receipt because deletion suppresses its completion message. Independently tracked live processes and other work still block closure.
+
+Search completions committed as atomic `custom_message` boundary entries drain their matching request before final settlement, even when Pi emits no `message_start`. A passive `web-search:cancelled` custom entry with `{ requestId, status: "cancelled", parentToolCallId? }` releases that request without waking a stopped model. A successful `wait_for_web_searches` also drains the known search set captured when that wait started, including searches cancelled during a provider retry. Failed waits, unrelated requests and searches launched after the wait started remain pending.
+
+Receipt and process ownership persists for the exact session through reload. Unreported daemons, unsupported native process identities and remote jobs without completion receipts are not discoverable by this guard. It does not adopt unrelated processes or idle runtime helpers.
+
+The settled handler returns its finalization promise. Pi must see the shutdown request before it checks that settled boundary; saving a result or starting detached finalization is not proof that the Pi process exited.
+
+The retained non-replica lifecycle is unchanged: autonomous clean or empty boundaries can record terminal intent after the last descendant result; provider errors and aborts keep those children open.
+
+Manual and automatic context compaction are active child work. `session_before_compact` protects the child from stale recovery until `session_compact` or `session_compact_failed` ends that state. Compaction never publishes or repeats a parent result.
+
+## Failed replica startup
+
+The parent records the allocated pane's shell incarnation before starting Pi. If extension loading or an earlier launch step fails, cleanup can use that launch record without waiting for child session initialization. Only a verified owned idle shell is closed; a changed boot, reused shell or live process blocks that cleanup. Keep failed source and session evidence.
+
+A terminal marker is process-exit evidence only after the replica process has stopped. Revalidate it after sidecar and diagnostic-read races. Report available startup diagnostics, not unrelated warnings or an inherited parent answer. A native launch error already saved in the exact parent counts as its outcome and is not replayed as another automatic result. A warning without process failure does not terminate a copy.
+
+## One terminal result
+
+A child publishes only when it reaches a terminal lifecycle outcome, such as auto-exit, cancellation, confirmed pane loss, or recovery failure. No model-facing completion or help tool is registered.
+
+The terminal path:
+
+1. reads the latest child result;
+2. waits for every recursive descendant to drain;
+3. writes one immutable result file and one durable inbox entry;
+4. materializes that inbox in the exact parent session once;
+5. records delivery before cleanup; and
+6. removes the child from active supervision as soon as delivery is recorded.
+
+The stable terminal delivery ID is the child ID prefixed with `terminal:`. Delivery uses an exclusive durable claim. A successful send is acknowledged once. A send that throws remains retryable. A send that Pi only queued — the streaming parent has not written it to its session yet — is never repeated: the claim stays held until the parent session holds the record, so one result cannot become many copies. Compaction, reload, restored watchers, and later parent tool calls cannot materialize an acknowledged delivery again.
+
+Old pending `settled` inbox entries are acknowledged without being sent. This prevents an upgrade or restored session from exposing an intermediate response after the terminal-only rule is active.
+
+Cleanup is separate from delivery. An ordinary child pane closes after delivery. A worktree child keeps its review workspace. A delivered child leaves the widget immediately; a rejected Herdr close retries a few times with backoff and then stops with a delivery-log record. Its result is not sent again.
+
+Replicas do not automatically restart after process loss or inactivity. Their parent receives a failure with available answer/error facts and can create a fresh copy. Normal settlement reports only after session work drains. Ten continuous seconds with Pi idle and no pending messages, movement or undrained descendants retry a missed completion. A saved settlement checkpoint survives reload; without settlement evidence, this safety net reports failure and preserves the answers. Active requests, tools, retries and compaction do not count as stopped time.
+
+Retained non-replica recovery is bounded to three attempts across crash and stale causes. An exhausted child is not killed, resumed or re-delivered on later watchdog passes.
+A recovery watcher starts only after the replacement registers its own session and pane. The old closed pane must not become a manual-cancellation result for the new process.
+
+## Descendants and exact-parent binding
+
+A child drains only after its terminal outcome reaches its exact immediate-parent inbox and all descendants drain recursively. Failed inbox publication has no timeout and is never rerouted. A pending session inbox stays bound to the stable parent session ID until that exact session returns, even through a changed file path or pane.
+
+Lineage identity belongs to the launched Pi session. A child lifecycle starts only after the runtime opens that session's persisted child binding. Reload startup checks, shell commands and test runners may inherit its environment variables, but that does not let them publish completion, update its activity or take ownership of its descendants. In particular, closing a no-session startup check must not report the live replica as closed.
+
+A public `SubagentTree` owner is different. Its callbacks process child results inside the tree runtime. The tree result still waits for all reserved descendants, but ordinary model-facing `subagent_result` delivery follows the one-terminal-result rule above.

package/docs/worktree-subagents.md

@@ -0,0 +1,58 @@
+# Worktree subagents
+
+The retained package-root API and internal launcher support opt-in isolated Git worktrees. This document describes those retained paths only. The model-facing `replicate` tool defaults to a source-snapshot worktree with automatic host-owned join and successful cleanup; it also supports a shared-directory mode. See the README for that separate contract.
+
+For example, a tree consumer can launch:
+
+```typescript
+tree.launch({
+  name: "issue-42-build",
+  agent: "implementer",
+  task: "Implement the approved change, test it, commit it, and report the SHA.",
+  worktree: { branch: "issue/42", base: "HEAD" },
+});
+```
+
+This example assumes an existing `implementer` role. Bare API children are also supported.
+
+`cwd` selects the source repository. `base` resolves to an exact commit before creation and defaults to committed `HEAD`. The child starts at the Herdr-created worktree root.
+
+## Guarantees
+
+1. Worktrees are opt-in per launch.
+2. Parent uncommitted and untracked files are not copied.
+3. An ownership manifest is written before Herdr creates resources.
+4. Herdr creates the workspace without changing parent focus.
+5. Success, failure, and help outcomes retain the workspace.
+6. Git inspection failures are reported as unknown, never guessed clean.
+7. The extension never pushes, merges, cherry-picks, switches the parent checkout, or removes worktrees and branches.
+
+The manifest lives under the parent session artifacts:
+
+```text
+<session-dir>/artifacts/<session-id>/worktree-runs/<child-id>.json
+```
+
+## Review
+
+After completion, inspect the reported path and Git state. Review the diff and commits before integrating. The parent owns merge, publication, and cleanup.
+
+Useful Git commands:
+
+```bash
+git -C <worktree-path> status --short
+git -C <worktree-path> log --oneline <base-sha>..HEAD
+git -C <worktree-path> diff <base-sha>...HEAD
+```
+
+## Recovery
+
+If creation returns an ambiguous Herdr error, the extension may reconcile one exact branch match from Herdr's worktree list. It never guesses among multiple matches. A failed launch retains any verified worktree and records the error in its manifest.
+
+The retained internal resume path opens an ordinary pane. It does not reattach ownership of a retained worktree.
+
+A parent or child may move through Teleport without transferring or discarding the original worktree ownership. The session's binding still resolves the launch manifest and pending answer. The new session location is used for lifecycle observation and delivery; the retained checkout remains the original launch's review target. Herdr process loss does not authorize closing an unrelated pane that reuses an old ID.
+
+## Cleanup
+
+Only remove a worktree after review and integration. Verify its path and branch first, then use your normal Git or Worktrunk cleanup procedure. Never delete `.git/worktrees` entries by hand.

package/examples/role-pack/extension.ts

@@ -0,0 +1,18 @@
+import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+import { fileURLToPath } from "node:url";
+
+const roles = fileURLToPath(new URL("./roles", import.meta.url));
+
+export default function rolePack(pi: ExtensionAPI) {
+	const unsubscribe = pi.events.on(
+		"pi-herdr-subagents:roles:discover:v1",
+		(request) => {
+			const discovery = request as {
+				apiVersion: number;
+				register(path: string): void;
+			};
+			if (discovery.apiVersion === 1) discovery.register(roles);
+		},
+	);
+	pi.on("session_shutdown", unsubscribe);
+}

package/examples/role-pack/package.json

@@ -0,0 +1,16 @@
+{
+  "name": "@example/pi-herdr-roles",
+  "version": "1.0.0",
+  "keywords": [
+    "pi-package"
+  ],
+  "type": "module",
+  "pi": {
+    "extensions": [
+      "./extension.ts"
+    ]
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*"
+  }
+}

package/examples/role-pack/roles/example-reviewer.md

@@ -0,0 +1,12 @@
+---
+description: Reviews a bounded change and reports concrete, evidence-backed findings
+tools: read, bash
+spawning: false
+auto-exit: true
+system-prompt: append
+---
+
+# Example Reviewer
+
+Review only the requested change. Report concrete findings with file and line
+references. Do not modify files.

package/index.d.ts

@@ -0,0 +1,116 @@
+import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
+export type ThinkingLevel = "off" | "minimal" | "low" | "medium" | "high" | "xhigh" | "max";
+export interface SubagentTreeMetadata {
+    readonly value: unknown;
+    /** Immutable package-owned identity; value is the consumer-owned opaque metadata. */
+    readonly callerId?: string;
+    readonly treeId?: string;
+    readonly ownerId?: string;
+    readonly nodeId?: string;
+    readonly parentId?: string;
+}
+export interface SubagentError {
+    readonly code: string;
+    readonly message: string;
+}
+export interface SubagentWorktree {
+    readonly path?: string;
+    readonly branch?: string;
+    readonly baseRef?: string;
+    readonly baseSha?: string;
+    readonly workspaceId?: string;
+    readonly paneId?: string;
+    readonly headSha?: string | null;
+    readonly commitsAhead?: number | null;
+    readonly clean?: boolean | null;
+    readonly conflicted?: boolean | null;
+    readonly changedFiles?: string[] | null;
+    readonly untrackedFiles?: string[] | null;
+    readonly manifestFile?: string;
+}
+export type SubagentChildOutcome = "clean" | "empty" | "error" | "intentional-abort" | "unexpected-abort";
+export interface SubagentChildResult {
+    readonly resultId: string;
+    readonly assistantEntryId: string;
+    readonly answer: string | null;
+    readonly outcome: SubagentChildOutcome;
+    readonly sessionReference?: string;
+    readonly error?: SubagentError;
+    readonly worktree?: SubagentWorktree;
+    readonly metadata?: SubagentTreeMetadata;
+}
+export interface SubagentNodeResult {
+    readonly nodeId: string;
+    readonly parentId: string;
+    readonly outcome: SubagentChildOutcome;
+    readonly open: boolean;
+    readonly sessionReference?: string;
+    readonly error?: SubagentError;
+}
+export interface SubagentTreeResult {
+    readonly terminalId: string;
+    readonly state: "completed" | "failed" | "cancelled";
+    readonly rootResult: SubagentChildResult;
+    readonly callerMetadata: SubagentTreeMetadata;
+    readonly nodes: readonly SubagentNodeResult[];
+    readonly error?: SubagentError;
+}
+export interface SubagentChildHandle {
+    readonly nodeId: string;
+    readonly parentId: string;
+    readonly result: Promise<SubagentChildResult>;
+}
+export interface LaunchChildOptions {
+    readonly parentId: string;
+    readonly name: string;
+    readonly task: string;
+    readonly agent?: string;
+    readonly skills?: string;
+    readonly tools: readonly string[];
+    readonly cwd?: string;
+    readonly worktree?: {
+        readonly branch: string;
+        readonly base?: string;
+    };
+    readonly metadata?: unknown;
+}
+export interface SubagentTreeHandle {
+    readonly callerId: string;
+    readonly treeId: string;
+    readonly ownershipToken: string;
+    readonly ownerId: string;
+    readonly children: ReadonlyMap<string, SubagentChildHandle>;
+    readonly result: Promise<SubagentTreeResult>;
+    launchChild(options: LaunchChildOptions): Promise<SubagentChildHandle>;
+    bindFinalCallback(callback: (result: SubagentTreeResult) => void | Promise<void>): void;
+    cancel(): Promise<SubagentTreeResult>;
+}
+export interface CreateSubagentTreeOptions {
+    readonly pi: ExtensionAPI;
+    readonly ctx: ExtensionContext;
+    readonly metadata?: unknown;
+}
+export interface ForkLineage {
+    readonly previousSessionFile: string;
+}
+export interface AttachSubagentTreeOptions {
+    readonly pi: ExtensionAPI;
+    readonly ctx: ExtensionContext;
+    readonly callerId?: string;
+    readonly treeId?: string;
+    readonly ownershipToken?: string;
+    readonly forkLineage?: ForkLineage;
+}
+export declare class SubagentTreeError extends Error {
+    constructor(message: string);
+}
+export declare class SubagentTreeValidationError extends SubagentTreeError {
+    constructor(message: string);
+}
+export declare class SubagentTreeOwnershipError extends SubagentTreeError {
+    constructor(message: string);
+}
+/** Create a caller-owned tree. Importing this module performs no registration. */
+export declare function createSubagentTree(options: CreateSubagentTreeOptions): SubagentTreeHandle;
+/** Reattach a caller or claim a child branch using its package context. */
+export declare function attachSubagentTree(options: AttachSubagentTreeOptions): SubagentTreeHandle;