@llblab/pi-actors 0.41.1 → 0.42.0

package/docs/recipe-library.md

@@ -13,14 +13,16 @@ Helper scripts that belong to library recipes live in root `scripts/`. The music
 
 ## Install Locally
 
-Recipes can be copied into the user recipe root:
+Select only the operator-facing recipe or wrapper you intend to own locally. For example:
 
 ```bash
 mkdir -p ~/.pi/agent/recipes
-cp <repo>/recipes/*.json ~/.pi/agent/recipes/
+cp <repo>/recipes/pipeline-review-readiness.json ~/.pi/agent/recipes/
 ```
 
-Or a registered tool can point directly at a recipe path when that is more convenient.
+Do not bulk-copy `recipes/*.json`. The packaged library also contains internal composition stages, including `draft-review.json` and `tool-review.json`; the automatic-review runtime launches those selectors with fenced inputs and they must not become user-installed callable tools.
+
+A registered tool can instead point at one selected recipe path when a durable operator-facing name is useful. Prefer a thin wrapper for public defaults or policy rather than copying the wrapper's internal imports.
 
 ## Async Subagent Components
 
@@ -32,6 +34,8 @@ Core subagent recipes:
 - `recipes/subagent-preflight.json`: Tiny model/thinking/tool-policy smoke check before expensive fanout; failures surface `ACTOR_PREFLIGHT_FAILED` with stage, selected policy, provider error class, prompt file, and override args.
 - Packaged reviewer, verifier, merger, judge, and normalizer stages use `accept_output: review_evidence` and require `ACTOR_REVIEW_RESULT` as the exact first non-whitespace output line. Marker prefixes, format acknowledgements, and input requests therefore remain rejected branch diagnostics rather than usable quorum evidence.
 - `recipes/subagent-review.json`: Evidence-grounded review lens.
+- `recipes/draft-review.json`: Internal no-tools selector for one immutable automatic draft batch. It receives an attached value-free structural projection with batch-local opaque occurrence/content-group identities, counts, risk labels, and usage—not canonical names, draft basenames, raw hashes, recipe bodies, template text, defaults, authored prose, or filesystem paths—then emits one terminal `DRAFT_REVIEW_RESULT` with quota-free promote/discard decisions. The executor derives any promotion from the separate trusted captured source.
+- `recipes/tool-review.json`: Internal no-tools selector for one immutable 36-tool portfolio. It receives the same identity-opaque value-free structural projection and may recommend quota-free keep, unchanged-source rename (`evolve`), unchanged-source demote, or identical-source merge decisions. `replace`, `split`, and returned recipe content fail mechanically; deterministic executors alone read trusted captured recipes and own validated safe-boundary activation.
 - `recipes/subagent-critic.json`: Assumption and failure-mode critique.
 - `recipes/subagent-plan.json`: Bounded plan slices and validation gates.
 - `recipes/subagent-evidence-map.json`: Evidence and confidence map.

package/docs/template-recipes.md

@@ -107,20 +107,13 @@ The high-priority user recipe directory is also the default tool set: recipes pl
 
 Higher-priority files shadow lower-priority files with the same basename. Within one priority layer, same-id JSON shadows Markdown because JSON is the canonical precise format. A highest-priority invalid recipe is still visible and blocks fallback so operators do not accidentally run packaged behavior when a user override is broken. A highest-priority recipe with `disabled: true` also blocks fallback, is not launchable, and intentionally disables that id. Healthy overrides are silent; failed bare-name launches caused by invalid or disabled shadowing include compact `reason=shadowed_invalid` or `reason=shadowed_disabled` diagnostics with the active path, blocked candidate, and recipe-doctor hint.
 
-## Usage Metadata
+## Usage And Lineage Metadata
 
-User-owned recipe launches may accumulate extension-maintained usage metadata in `.usage/<recipe-filename>.json` sidecars:
+User-owned recipe launches update extension-maintained lineage ledgers under `.usage/recipes/<recipe-name>.json` plus a path index. Authored recipes remain untouched. The ledger keeps lifetime and revision-local launch counts, executable fingerprints, former names and paths, bounded revision ancestry, transition events, and review epochs.
 
-```json
-{
-  "calls": 12,
-  "last_called": "2026-05-22T10:30:00.000Z"
-}
-```
-
-The extension increments `calls` and updates `last_called` when it starts that concrete recipe, either through a recipe-backed tool call or a direct async recipe-file run. The sidecar also stores a content `fingerprint`; if authored recipe content changes, the next launch resets `calls` before counting the new launch and records `reset_at`. Keeping telemetry outside the recipe prevents usage writes from replacing concurrent operator edits; discovery merges sidecar usage into inspection. Agents should treat these fields as cleanup evidence, not as authored recipe contract. Packaged standard-library recipes do not receive usage metadata.
+Lifetime usage survives rename, promotion, demotion, and executable revision. Revision-local counters restart when executable content changes, making the new fingerprint eligible for portfolio review without erasing prior evidence. Discovery merges current lineage evidence into inspection; packaged standard-library recipes receive no mutable usage ledger. The unreleased format stays intentionally unversioned until a public compatibility boundary exists.
 
-There is intentionally no failure counter in the recipe contract. A failed launch can reflect caller misuse, missing runtime values, or an environmental problem rather than recipe uselessness. Cleanup decisions should be explicit operator work: keep as a tool, move out of the agent recipe root to retain recipe-only memory, merge, delete, or archive.
+There is intentionally no failure counter: a failed launch can reflect caller misuse, missing values, or environment state rather than recipe quality. Usage remains evidence, not an automatic usefulness verdict. Automatic review combines it with contract quality, portability, duplication, safety, and likely future value. `register_tool draft=...` is the preferred fenced single-draft override; a deliberate move/copy from `drafts/` into the recipe root also remains valid, though it may defer an already captured automatic batch.
 
 For object form, keep `template` last. Recipe metadata comes first; executable content stays last.
 

package/docs/tool-registry.md

@@ -10,13 +10,15 @@ The registry source is location-discovered recipes, not a live tool-only JSON fi
 
 - `~/.pi/agent/recipes/*.json` and `*.md` are the highest-priority user recipe root and the operator-managed tool set.
 - Recipes in that root are tools by location.
-- `~/.pi/agent/recipes/drafts/*.json` stores captured inline-spawn draft recipes, not registered tools. Promote one with `register_tool name=<tool_name> draft=<draft_path>` or by manually moving/copying it up one level into `~/.pi/agent/recipes`. `inspect target=recipes view=summary` reports their count, and verbose output lists their paths, timestamps, fingerprints, validation state, source run when known, descriptions, and template previews for explicit replay or promotion.
+- `~/.pi/agent/recipes/drafts/*.json` stores captured inline-spawn draft recipes, not registered tools. Twelve drafts trigger one silent automatic exact-batch review after the foreground turn and active actors finish; the deterministic executor promotes or discards every reviewed source while preserving newer drafts for the next batch. Promote one earlier with the fenced `register_tool name=<tool_name> draft=<draft_path>` override or a deliberate move/copy into the recipe root. A direct filesystem promotion remains legitimate, but it can shrink or invalidate a captured batch and defer its automatic cleanup; lineage reattaches on the next launch when the move remains unambiguous. No manual batch-consolidation command exists. `inspect target=recipes view=summary` reports draft count, and verbose output lists paths, timestamps, fingerprints, validation state, source run when known, descriptions, and template previews.
 - Packaged pi-actors recipes are the lower-priority standard library of declarative actor config components, not automatically registered tools.
 - Ad hoc recipe files outside the user recipe root are components unless explicitly registered/copied into `~/.pi/agent/recipes`.
-- Recipe identity is the filename basename; `~/.pi/agent/recipes/docs_review.json` and `docs_review.md` both have id/tool name `docs_review`.
+- The current tool name is the filename basename; `~/.pi/agent/recipes/docs_review.json` and `docs_review.md` both expose `docs_review`. A canonical name-and-priority lineage ledger preserves usage and revision history as draft/active location changes; controlled rename transfers that history to the new name and retains the former name.
+- Set `PI_ACTORS_AUTOMATIC_REVIEW=off` (also accepts `false` or `0`) before starting Pi to disable draft/tool reviewer scheduling and safe-boundary portfolio activation; runtime status exposes the effective policy.
+- Active user recipes receive zero-call lineage without fabricating launches. Once thirty-six non-sensitive current revisions lack a review fingerprint, pi-actors captures the oldest exact portfolio and silently attaches an identity-opaque value-free structural projection with batch-local equality-only content groups to the no-tools `tool-review` actor after a foreground turn and only while no other actor runs. A content revision resets revision-local usage and becomes eligible again while lifetime usage remains continuous. The reviewer may select only `keep`, unchanged-source rename (`evolve`), unchanged-source `demote`, or `merge` for canonically identical captured recipes; it never returns recipe content. `replace`, `split`, and executable contract changes require explicit operator authoring. Completed output becomes an immutable size-bounded approval plan only after exact result, source-hash, target-collision, and lineage-projection validation; approval itself never mutates active recipes. At the next `session_start`, filesystem commit persists `lineage_pending`, journaled lineage records roll forward, `completed` persists, and only then may quarantine be removed before runtime tool discovery.
 - Same-id JSON shadows Markdown in the same priority layer.
 
-Because the user recipe directory is sticky agent muscle memory, runtime launches update `usage.calls`, `usage.last_called`, and a content `usage.fingerprint` in `.usage/<recipe-filename>.json` sidecars rather than rewriting authored recipe files. If authored recipe content changes, the next launch resets `usage.calls` and records `usage.reset_at` before counting the launch, so usage evidence follows the current recipe meaning without racing operator edits. Discovery and file-watcher refresh merge sidecar usage into inspect summaries. `inspect target=recipes view=summary verbose=true` includes usage metadata and operator-gated cleanup recommendations for invalid, shadowed, disabled, component-only, unused, or overriding recipes. Recommended actions stay explicit: keep as a tool/component, enable, merge, fix, delete, or archive. The extension does not maintain a failure counter and agents should not silently clean tools during unrelated work.
+Because the user recipe directory is sticky agent muscle memory, runtime launches update a stable lineage ledger under `.usage/recipes/<recipe-name>.json` plus a priority-compatible path index rather than rewriting authored recipe files. Launch accounting briefly shares the canonical recipe-root fence used by portfolio activation before taking index/ledger locks; activation therefore cannot quarantine a source between launch authorization and accounting. If activation already changed the loaded source, the stale invocation rejects with a reload-and-retry error instead of executing without usage evidence. `lifetime_calls` and the compatibility `calls` view survive rename, promotion, demotion, and content revision; `revision_calls` restarts only when the executable fingerprint changes. The bounded ledger retains former paths/names, revision ancestry, promotion/demotion events, and review epochs. An unambiguous external rename follows its prior lineage by fingerprint. Because automatic review has not shipped publicly, its inputs, results, admission state, plans, journals, evidence, lineage storage, and snapshots remain unversioned rather than carrying migration branches for discarded internal iterations. Discovery and file-watcher refresh merge ledger usage into inspect summaries. `inspect target=recipes view=summary verbose=true` includes usage metadata and operator-gated cleanup recommendations for invalid, shadowed, disabled, component-only, unused, or overriding recipes. The extension does not maintain a failure counter.
 
 `register_tool` is the preferred agent-facing mutation API. It creates, updates, and deletes recipe files in `~/.pi/agent/recipes`; agents do not need to edit the files directly for normal registration. Extension-authored register, update, delete, draft-promotion, and usage-metadata mutations hold a cross-process lock keyed by filesystem-canonical recipe identity across the complete check/read/write/runtime-update window. Existing targets or the nearest existing parent are resolved through `realpath`, so real and symlink aliases serialize while unrelated recipes remain independent; stale locks are reclaimed only after their owner is proven dead. Direct file edits are still valid for operators and advanced agents. Runtime behavior is reactive: file creation, deletion, or edits in the user recipe root trigger validation and tool-set refresh, with invalid recipes surfaced as diagnostics rather than silently ignored. If the recipe root does not exist at session start, an advisory parent watcher detects its creation and switches to the normal root watcher; deletion or rename rearms the parent watcher without polling.
 
@@ -27,10 +29,15 @@ inspect target=tool:pi-actors view=status
 inspect target=tool:pi-actors view=triage
 inspect target=recipes view=status
 inspect target=recipes view=doctor
+inspect target=recipes view=reviews
 inspect target=recipes view=summary verbose=true
 ```
 
-`tool:pi-actors` is a reserved runtime-status actor. `view=status` reports the loaded package version, package root, source/dist mode, entrypoint path, recipe roots, and git commit when available. Use it after reloads to confirm which extension code is actually live. `view=triage` adds a compact attention surface for active runs, other-session runs, invalid or blocking recipes, exposed tool recipes with non-lifecycle risk labels, drafts, stale claims, failed runs, attention messages, and next inspect actions without repairing anything. Packaged components and recipes whose only label is `risk.long_running` stay in recipe doctor/summary evidence rather than triage attention.
+`inspect target=recipes view=reviews` returns bounded read-only evidence for automatic draft/tool review phases, decision counts, garbage collection, lineage revisions, demotions, rollback provenance, retained revision snapshots, and bounded `failed_stage`/`last_error`/`next_action` fields. It never starts a review or generates a follow-up turn. Automatic reviewers receive only an attached value-free projection—counts, risk labels, bounded usage, and command-graph shape without recipe bodies, template/default values, authored prose, or filesystem paths—and no general filesystem or mutation tools. Internal snapshot rollback writes one CAS-authenticated journal before changing either recipe or lineage state; interruption after either write rolls forward on the next identical rollback request instead of returning a permanently split recipe/ledger state.
+
+Explicit recovery stays inside the existing actor-message surface: send `review.retry` or `review.reset` to `tool:pi-actors` with `body={"scope":"draft"}` or `body={"scope":"tool"}`. Retry resets bounded launch/processing counters and reuses the immutable batch. If a draft transaction journal already exists, retry preserves the original reviewer run and resumes that authenticated journal plan; even changed reviewer stdout cannot redirect committed recipe or lineage targets. When a tool transaction already committed, retry preserves approval/transaction evidence and returns to the safe activation/lineage boundary rather than launching another reviewer. Reset removes only disposable failed/completed admission state and rejects tool cycles that still carry recovery evidence.
+
+`tool:pi-actors` is a reserved runtime-status/control actor. `view=status` reports the loaded package version, package root, source/dist mode, entrypoint path, recipe roots, automatic-review policy, and git commit when available. Use it after reloads to confirm which extension code is actually live. `view=triage` adds a compact attention surface for active runs, other-session runs, invalid or blocking recipes, exposed tool recipes with non-lifecycle risk labels, drafts, stale claims, failed runs, attention messages, and next inspect actions without repairing anything. Packaged components and recipes whose only label is `risk.long_running` stay in recipe doctor/summary evidence rather than triage attention.
 
 The recipe summary reports active, shadowed, invalid, disabled, and diagnostic entries so operators can answer why a tool is present, hidden, broken, or disabled. The doctor view keeps the same registry evidence but promotes an advisory action surface: compact output includes the highest-priority `top` remediation, risk-label counts, and ordered actions for invalid/blocking, disabled, risky shell-boundary, and shadowed recipes. Verbose inspection keeps per-recipe `risk_labels`, the structured `risk_summary`, `remediations`, `top_action`, diagnostic details, and blocked lower-priority fallback paths when a broken or disabled higher-priority recipe masks a fallback. Risk labels are deterministic review aids, not execution blockers or sandbox claims.
 

package/index.ts

@@ -5,118 +5,32 @@
  * Wraps command templates as callable pi tools, stores durable user tools as recipe files, and exposes actor orchestration across reloads and sessions.
  */
 
-import * as AsyncRuns from "./lib/async-runs.ts";
+import * as AutomaticReviewRuntime from "./lib/automatic-review-runtime.ts";
 import * as CommandTemplates from "./lib/command-templates.ts";
-import * as InspectorOverlay from "./lib/inspector-overlay.ts";
-import * as Observability from "./lib/observability.ts";
+import * as InspectorCommand from "./lib/inspector-command.ts";
 import * as Paths from "./lib/paths.ts";
 import * as Pi from "./lib/pi.ts";
 import * as Prompts from "./lib/prompts.ts";
+import * as RunUiRuntime from "./lib/run-ui-runtime.ts";
 import * as Runtime from "./lib/runtime.ts";
 import * as Temp from "./lib/temp.ts";
 import * as Tools from "./lib/tools.ts";
 import * as ToolsResponse from "./lib/tools-response.ts";
 
 export default function toolRegistryExtension(pi: Pi.ExtensionAPI) {
-  let runsAnimationInterval: NodeJS.Timeout | undefined;
-  let runsNotifyTimeout: NodeJS.Timeout | undefined;
   let activeRunContext: Pi.ExtensionContext | undefined;
-  let lastRunWatcherDiagnosticId = 0;
-  const runUi = Observability.createRunUiObservationState();
-  const retirementAttempts = new Set<string>();
-  const terminalNotificationsInFlight = new Set<string>();
   const getRunOwnerId = Pi.getSessionId;
-  const retireCandidateRuns = (
-    ctx: Pi.ExtensionContext,
-    summary: Observability.RunSummary,
-  ): void => {
-    void Observability.executeRunRetirements(summary, {
-      attempted: retirementAttempts,
-      cancelRun: (candidate) => AsyncRuns.cancelRun(candidate.stateDir),
-      notify: (message, level) => ctx.ui.notify(message, level),
-      sendStop: (candidate) =>
-        AsyncRuns.sendRunMessage(candidate.stateDir, "stop"),
-    });
-  };
-  const updateRunUi = (
-    ctx: Pi.ExtensionContext,
-    notify = false,
-    terminalOnly = false,
-  ): void => {
-    const ownerId = getRunOwnerId(ctx);
-    const snapshot = Observability.readRunUiSnapshot(runUi, ownerId);
-    ctx.ui.setStatus(
-      "zz-pi-actors-runs",
-      snapshot.status ? ctx.ui.theme.fg("dim", snapshot.status) : undefined,
-    );
-    if (!notify) return;
-    const notificationSink = Pi.createNotificationSink(pi, ctx);
-    retireCandidateRuns(ctx, snapshot.summary);
-    Observability.deliverRunTransitionNotifications(
-      snapshot.transitions,
-      notificationSink,
-      terminalNotificationsInFlight,
-    );
-    Observability.pruneRunUiObservationState(runUi, snapshot);
-    if (!terminalOnly) {
-      Observability.deliverRunOutboxNotifications(
-        snapshot.outboxEvents,
-        notificationSink,
-      );
-    }
-  };
-  const closeRunWatchers = (): void => {
-    runWatcher.close();
-    terminalReconciliation.close();
-    if (runsNotifyTimeout) clearTimeout(runsNotifyTimeout);
-    runsNotifyTimeout = undefined;
-  };
-  const reportRunWatcherDiagnostics = (ctx: Pi.ExtensionContext): void => {
-    for (const diagnostic of runWatcher.getDiagnostics()) {
-      if (diagnostic.id <= lastRunWatcherDiagnosticId) continue;
-      lastRunWatcherDiagnosticId = diagnostic.id;
-      ctx.ui.notify(
-        diagnostic.message,
-        diagnostic.code === "rearmed" ? "info" : "warning",
-      );
-    }
-  };
-  const scheduleRunEventUpdate = (): void => {
-    if (runsNotifyTimeout) clearTimeout(runsNotifyTimeout);
-    runsNotifyTimeout = setTimeout(() => {
-      const ctx = activeRunContext;
-      if (!ctx) return;
-      runWatcher.refresh();
-      updateRunUi(ctx, true);
-      reportRunWatcherDiagnostics(ctx);
-    }, 50);
-    runsNotifyTimeout.unref?.();
-  };
-  const runWatcher = Observability.createRunStateWatcher({
-    stateRoot: Paths.EXTENSION_RUNTIME_PATHS.runStateRoot,
-    onChange: scheduleRunEventUpdate,
+  const automaticReview = AutomaticReviewRuntime.createAutomaticReviewRuntime({
+    getActiveContext: () => activeRunContext,
+    getRunOwnerId,
+    getThinkingLevel: () => pi.getThinkingLevel(),
+  });
+  const runUiRuntime = RunUiRuntime.createRunUiRuntime({
+    getActiveContext: () => activeRunContext,
+    getRunOwnerId,
+    onRunEvent: automaticReview.schedule,
+    pi,
   });
-  const terminalReconciliation =
-    Observability.createRunTerminalReconciliationLoop({
-      onError: (error) => {
-        const ctx = activeRunContext;
-        if (!ctx) return;
-        const message = error instanceof Error ? error.message : String(error);
-        ctx.ui.notify(`Actor terminal reconciliation failed: ${message}`, "error");
-      },
-      reconcile: () => {
-        const ctx = activeRunContext;
-        if (!ctx) return;
-        Observability.reconcileRunTerminalNotifications({
-          inFlight: terminalNotificationsInFlight,
-          ownerId: getRunOwnerId(ctx),
-          sink: Pi.createNotificationSink(pi, ctx),
-          state: runUi,
-        });
-        reportRunWatcherDiagnostics(ctx);
-      },
-      refreshWatcher: () => runWatcher.refresh(),
-    });
   const actorToolDefinitions = new Map<string, Tools.ActorToolDefinition>();
   const withCurrentThinkingContext = <T extends Tools.ActorToolDefinition>(
     definition: T,
@@ -164,54 +78,26 @@ export default function toolRegistryExtension(pi: Pi.ExtensionAPI) {
     // Clear the pre-overlay widget after hot reloads from older pi-actors builds.
     ctx.ui.setWidget("zz-pi-actors-comms", undefined);
     activeRunContext = ctx;
-    closeRunWatchers();
+    runUiRuntime.close();
+    automaticReview.close();
     recipeReload.close();
     await Temp.prepareExtensionTempDir(Paths.EXTENSION_RUNTIME_PATHS.tempDir);
     if (activeRunContext !== ctx) return;
+    automaticReview.start(ctx);
     runtime.loadTools(ctx);
-    updateRunUi(ctx, true, true);
-    runWatcher.refresh();
-    terminalReconciliation.start();
+    runUiRuntime.start(ctx);
     recipeReload.watch(ctx);
-    if (runsAnimationInterval) clearInterval(runsAnimationInterval);
-    runsAnimationInterval = setInterval(() => {
-      if (activeRunContext === ctx) updateRunUi(ctx, false);
-    }, 1000);
-    runsAnimationInterval.unref?.();
   });
-  pi.on("session_shutdown", async () => {
-    if (runsAnimationInterval) clearInterval(runsAnimationInterval);
-    runsAnimationInterval = undefined;
+  pi.on("agent_end", async (_event, ctx) => {
+    if (activeRunContext === ctx) automaticReview.schedule();
+  });
+  pi.on("session_shutdown", async (event, ctx) => {
     activeRunContext = undefined;
-    closeRunWatchers();
+    automaticReview.close();
     recipeReload.close();
+    runUiRuntime.shutdown(event.reason, ctx);
   });
-  pi.registerCommand("actors-inspector", {
-    description: "Open the keyboard-driven actor inspector overlay",
-    handler: async (_args, ctx) => {
-      ctx.ui.setWidget("zz-pi-actors-comms", undefined);
-      await ctx.ui.custom<void>(
-        (tui, theme, _keybindings, done) =>
-          new InspectorOverlay.ActorInspectorOverlay({
-            done,
-            ownerId: getRunOwnerId(ctx),
-            stateRoot: Paths.EXTENSION_RUNTIME_PATHS.runStateRoot,
-            theme,
-            tui,
-          }),
-        {
-          overlay: true,
-          overlayOptions: {
-            anchor: "center",
-            width: "94%",
-            minWidth: 72,
-            maxHeight: "94%",
-            margin: 1,
-          },
-        },
-      );
-    },
-  });
+  InspectorCommand.registerActorInspectorCommand(pi, getRunOwnerId);
   pi.on("before_agent_start", async (event) => ({
     systemPrompt: `${event.systemPrompt}\n\n${Prompts.ONBOARDING_SYSTEM_PROMPT}`,
   }));
@@ -221,11 +107,10 @@ export default function toolRegistryExtension(pi: Pi.ExtensionAPI) {
       configPath: Paths.EXTENSION_RUNTIME_PATHS.configPath,
       getActiveTools: () => pi.getActiveTools(),
       getRuntimeTool: (name) =>
-        Tools.resolveActiveRuntimeTool(
-          name,
-          runtime.getTools(),
-          (activeName) => actorToolDefinitions.get(activeName),
+        Tools.resolveActiveRuntimeTool(name, runtime.getTools(), (activeName) =>
+          actorToolDefinitions.get(activeName),
         ),
+      handleRuntimeMessage: automaticReview.handleMessage,
       registryRuntime: runtime,
       setActiveTools: (toolNames) => pi.setActiveTools(toolNames),
     }).map(withCurrentThinkingContext),