@introspection-ai/operator 0.1.0 → 0.1.2

package/README.md

@@ -9,19 +9,31 @@ The Operator is the agent that builds, deploys and improves other agents (Recipe
 | `evals/` | The evals that measure each loop (to come) |
 | `docs/` | [`DESIGN.md`](docs/DESIGN.md): what the Operator is being rebuilt to, and the decisions still open |
 
-## Run build locally
+## Run build
+
+You need Node 24, [uv](https://docs.astral.sh/uv/) and credentials for the model in `agents/build.yaml` (sign in once with `/login` in Pi).
 
 ```sh
-pnpm build                                       # a new agent, in ~/.introspection/agents/new-<n>
-pnpm build ~/.introspection/agents/new-1         # resume that build
-pnpm build ~/work/my-agent --model <model>       # another folder; flags after it go to Pi
+npm i -g @introspection-ai/cli && introspection setup   # once: Pi and the Recipes release it needs
+npx @introspection-ai/operator                          # a new agent, in ~/.introspection/agents/new-<n>
+npx @introspection-ai/operator ~/work/my-agent          # start or resume a build in that folder
+npx @introspection-ai/operator ~/work/my-agent --model <model>   # flags after the folder go to Pi
 ```
 
-`scripts/build.sh` starts the build agent (`pi --recipe <this repo> --agent build --no-context-files`) in a folder of its own, because build treats the folder it starts in as the owner's workspace: it creates the agent's repository there and resumes from what it finds there. It turns off Pi's loading of AGENTS.md and CLAUDE.md from that folder and the folders above it, which are written for coding agents and can describe other agents build must not copy. It refuses a folder inside this repository or nested in another git repository, installs this repository's dependencies with pnpm once, and checks that Pi's `@introspection-ai/recipes` satisfies the `peerDependencies` range, printing the `pi install` command to run when it doesn't. Set `INTROSPECTION_AGENTS_DIR` to put new builds somewhere other than `~/.introspection/agents`. You need Pi and the `introspection` CLI (`npm i -g @introspection-ai/cli`), and credentials for the model in `agents/build.yaml`.
+`npx @introspection-ai/operator` runs the package's launcher, `bin/introspection-operator.js`. It starts the build agent (`pi --recipe <the package> --agent build --no-context-files -e bin/local-workspace.mjs`) in a folder of its own, because build treats the folder it starts in as the owner's workspace: it creates the agent's repository there and resumes from what it finds there.
+- **No AGENTS.md or CLAUDE.md:** it turns off Pi's loading of those files from that folder and the folders above it. They're written for coding agents and can describe other agents, which build must not copy.
+- **Folder safety:** it refuses a folder inside the package or nested in another git repository; the agent's own repository is allowed, to resume.
+- **Python:** it builds the Recipe's Python environment (`package.json#pi.runtime.python`: paper-docx, pypdf, pypdfium2 and pillow, locked in `python/uv.lock`) with `uv sync --frozen` into `~/.introspection/operator-python/<version>-<lockfile hash>` (outside the package, which a global install may not let it write to; rebuilt only when the pins change), checks the declared imports, and starts Pi with it as `VIRTUAL_ENV`, so the `documents` tool can read and mark up Word and PDF files. Pi doesn't build `pi.runtime.python` itself.
+- **Shared documents:** it creates `files/` and `outputs/` in the build folder, beside the agent's repository, and loads `bin/local-workspace.mjs`, which tells the agent where they are, as the cloud does with `/workspace/files` and `/workspace/outputs`. Put the documents you'll share in `files/`: the documents tool opens files only inside the build folder. Given a path elsewhere, the agent copies the file into `files/` first.
+- **Versions:** it checks that Pi's `@introspection-ai/recipes` satisfies the `peerDependencies` range, and says to run `introspection upgrade` when it doesn't.
+
+Set `INTROSPECTION_AGENTS_DIR` to put new builds somewhere other than `~/.introspection/agents`.
+
+In a checkout of this repository, `pnpm build` runs the same launcher, and installs the dependencies with pnpm once.
 
 ## Package
 
-`@introspection-ai/operator` is the npm package published from this repository's root. It contains the Recipe a person runs (`SYSTEM.md`, `agents/` and `skills/`), so `pi install npm:@introspection-ai/operator` then `pi --recipe @introspection-ai/operator --agent build` starts a build, and `npx skills sync` finds the skills in it. `docs/`, `scripts/` and the evals are never published.
+`@introspection-ai/operator` is the npm package published from this repository's root. It contains the Recipe a person runs (`SYSTEM.md`, `agents/`, `skills/`, and `python/pyproject.toml` and `uv.lock` for its documents tool) and its launcher (`bin/`), so `npx @introspection-ai/operator` starts a build, and `npx skills sync` finds the skills in it. `docs/`, `scripts/` and the evals are never published.
 
 Skills are grouped by the stage of an agent's life (see [`docs/DESIGN.md`](docs/DESIGN.md)), and the group is the prefix of each skill's name, because `npx skills sync` finds skills only at `skills/<name>/SKILL.md` and not in nested folders:
 
@@ -33,8 +45,8 @@ Skills are grouped by the stage of an agent's life (see [`docs/DESIGN.md`](docs/
 | `optimize` | `optimize-quality`, `optimize-cost`, `optimize-latency` | Raise quality, then cut cost, then latency, each holding the gains of the one before | Designed; replaces today's `optimize` |
 | `harden` | `harden-scan`, `harden-spar`, `harden-stress` | Last, and optional apart from the scan: check the Recipe against the OWASP Agentic Skills Top 10, spar with it as a red team, and test it under load; failures become permanent cases | Designed; new, with no workflow behind it today |
 
-Releases are cut by release-please (`.github/workflows/release.yml`): Conventional Commit titles on `main` update a release pull request, and merging it tags `operator-vX.Y.Z` and publishes to npm through trusted publishing, starting at 0.1.0. CI runs `node scripts/check-skills.mjs` (frontmatter, folder names, and that exactly the Recipe is packed: `skills/`, `agents/` and `SYSTEM.md`) and `npm pack --dry-run`; publishing also requires at least one skill.
+Releases are cut by release-please (`.github/workflows/release.yml`): Conventional Commit titles on `main` update a release pull request, and merging it tags `operator-vX.Y.Z` and publishes to npm through trusted publishing, starting at 0.1.0. CI runs `node scripts/check-skills.mjs` (frontmatter, folder names, and that exactly the Recipe and its launcher are packed: `skills/`, `agents/`, `SYSTEM.md`, `python/pyproject.toml`, `python/uv.lock` and `bin/introspection-operator.js`) and `npm pack --dry-run`; publishing also requires at least one skill.
 
 ## License
 
-What `@introspection-ai/operator` publishes (`SYSTEM.md`, `agents/` and `skills/`) is licensed under the Apache License 2.0; see [`skills/LICENSE`](skills/LICENSE). Everything else in this repository is proprietary to Introspection and not licensed for use outside it.
+What `@introspection-ai/operator` publishes (`SYSTEM.md`, `agents/`, `skills/` and `bin/`) is licensed under the Apache License 2.0; see [`skills/LICENSE`](skills/LICENSE). Everything else in this repository is proprietary to Introspection and not licensed for use outside it.

package/SYSTEM.md

@@ -4,7 +4,7 @@ Work in loops, and keep the work in the agent's repository, not in the conversat
 
 Owners are usually business owners, not engineers. Speak plainly, ask only what changes what the agent does, and show a concrete example when a question would be abstract. An owner's approval is an explicit message from them, never inferred.
 
-Documents the owner gives you are of two kinds. The owner's own standing material (their playbook, guidelines, standard terms, templates) is meant to be built into the agent: read it when they give it to you and commit it with the Recipe. Material that arrives with a request (a customer's contract, an inbox) is a case input: read it, refer to it by content hash, and never commit it.
+When you need a document from the owner, ask them to share it: as an upload, unless the workspace says how this owner shares files. Documents the owner gives you are of two kinds. The owner's own standing material (their playbook, guidelines, standard terms, templates) is meant to be built into the agent: read it when they give it to you and commit it with the Recipe. Material that arrives with a request (a customer's contract, an inbox) is a case input: read it where it was shared, refer to it by content hash, and never commit it or copy it into the agent's repository.
 
 To read or show a Word or PDF document, use the `documents` tool: it reads text, tables and comments, renders scanned pages as images, and for Word makes tracked changes and comments, so you can show the owner a real example markup on their own contract. Without it, note that Pi's `read` tool handles text and images but returns Word and PDF files as raw bytes. Extract their text instead, preferring what's installed: `pdftotext` for PDF (the platform image always has it) and an installed `python-docx` for Word. Only when developing locally without them, use `uv run --with pypdf python` or `uv run --with python-docx python`; on the platform nothing is downloaded while you run. Never call a document unreadable or image-based without trying text extraction first; if it truly has no text, say so plainly and ask for another copy.
 

package/agents/build.yaml

@@ -3,7 +3,7 @@ description: Builds a new agent with its owner, from first idea to a pull reques
 ai:
   model: openai/gpt-6-luna
   thinking_level: medium
-tools: [read, bash, edit, write, ask_user_question, documents]
+tools: [read, bash, edit, write, ask_user_question, documents, run_case, case_status, stop_case]
 skills: [build-brainstorm, build-draft, build-review]
 system_instructions:
   mode: append
@@ -30,7 +30,7 @@ system_instructions:
     question per call: a short header, the question, and two or three option
     labels without numbers (the tool numbers them and always lets the owner
     type their own answer). Don't list the options in the question text. When
-    the answer must be free text, such as a file path, ask it in your reply
+    the answer must be free text, or files the owner shares, ask in your reply
     instead. Write each question so it works read aloud: under about 75 words,
     one decision, with enough context that a bare option number still makes
     sense. Without a question tool, write up to three numbered options in the
@@ -39,8 +39,10 @@ system_instructions:
     the step; name the step only in its first message. Never re-ask something
     answered.
 
-    When the owner gives you a path to a document, read it straight away;
-    don't ask permission or ask for a summary instead.
+    Read the documents the owner shares straight away, from the inputs
+    folder the workspace names; don't ask permission or ask for a summary
+    instead. Write what you make for the owner, such as an example result on
+    their sample, to the workspace's outputs folder.
 
     At the start of every session, find out where the work is before asking
     anything, looking only in the current folder and the folders inside it.

package/bin/introspection-operator.js

@@ -0,0 +1,150 @@
+#!/usr/bin/env node
+// Start the Operator's build agent in a folder of its own.
+//
+//   npx @introspection-ai/operator                       new build in ~/.introspection/agents/new-<n>
+//   npx @introspection-ai/operator <folder>              start, or resume, a build in <folder>
+//   npx @introspection-ai/operator [<folder>] <pi flags> pass flags such as --model on to Pi
+//
+// In a checkout of this repository, `pnpm build` runs this same script.
+//
+// Build treats the folder it starts in as the owner's workspace: it creates the agent's repository
+// there and resumes from what it finds, so it must not start inside this package or another repository.
+import { execFileSync, spawnSync } from "node:child_process";
+import { createHash } from "node:crypto";
+import { existsSync, mkdirSync, readFileSync, realpathSync } from "node:fs";
+import { homedir } from "node:os";
+import { delimiter, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
+const pkg = JSON.parse(readFileSync(join(root, "package.json"), "utf8"));
+const agentsDir = process.env.INTROSPECTION_AGENTS_DIR || join(homedir(), ".introspection", "agents");
+const piDir = process.env.PI_CODING_AGENT_DIR || join(homedir(), ".pi", "agent");
+
+function fail(message) {
+  console.error(`introspection-operator: ${message}`);
+  process.exit(1);
+}
+
+let args = process.argv.slice(2);
+if (args[0] === "--help" || args[0] === "-h") {
+  console.log(readFileSync(fileURLToPath(import.meta.url), "utf8").split("\n").slice(1, 8).map((l) => l.replace(/^\/\/ ?/, "")).join("\n"));
+  process.exit(0);
+}
+if (args[0] === "--") args = args.slice(1); // pnpm passes a separating `--` through
+
+// A package manager puts this package's node_modules/.bin first on PATH, and the Recipes package installs
+// its own Pi there, one without the Recipe extension. Use the Pi the owner installed.
+const ownBin = join(root, "node_modules", ".bin");
+const path = (process.env.PATH || "").split(delimiter).filter((dir) => resolve(dir) !== ownBin).join(delimiter);
+const env = { ...process.env, PATH: path };
+const has = (cmd) => spawnSync(cmd, ["--version"], { env, stdio: "ignore", shell: process.platform === "win32" }).status === 0;
+if (!has("pi")) fail("Pi is not installed. Run: npm i -g @introspection-ai/cli && introspection setup");
+
+// The Recipe's own dependencies (the documents tool) load from this package by Node's resolution, which
+// also finds them where npx and npm hoist them, beside the package. A fresh checkout installs them once.
+// Search node_modules folders upward from the package, as Node does; resolving `<name>/package.json`
+// would fail for a package whose exports don't list it.
+const resolvable = (name) => {
+  for (let dir = root; ; dir = dirname(dir)) {
+    if (existsSync(join(dir, "node_modules", name, "package.json"))) return true;
+    if (dirname(dir) === dir) return false;
+  }
+};
+if (!Object.keys(pkg.dependencies ?? {}).every(resolvable)) {
+  if (!existsSync(join(root, "pnpm-lock.yaml"))) fail(`can't find the Operator's dependencies from ${root}; install it with npm or npx`);
+  console.error("introspection-operator: installing the Operator's dependencies (once)");
+  execFileSync("pnpm", ["install", "--frozen-lockfile", "--reporter=silent"], { cwd: root, env, stdio: "inherit", shell: process.platform === "win32" });
+}
+
+// Pi loads the Recipe through its own copy of @introspection-ai/recipes, which must satisfy the peer range.
+const range = pkg.peerDependencies?.["@introspection-ai/recipes"] ?? "";
+const min = /^>=\s*(\d+\.\d+\.\d+)/.exec(range)?.[1];
+const piRecipes = join(piDir, "npm", "node_modules", "@introspection-ai", "recipes", "package.json");
+if (min) {
+  if (!existsSync(piRecipes)) fail("Pi has no @introspection-ai/recipes package. Run: introspection setup");
+  const have = JSON.parse(readFileSync(piRecipes, "utf8")).version;
+  const num = (v) => v.split(/[.-]/).slice(0, 3).map(Number);
+  const [a, b] = [num(have), num(min)];
+  if ((a[0] - b[0] || a[1] - b[1] || a[2] - b[2]) < 0) {
+    fail(`Pi uses @introspection-ai/recipes ${have}; the Operator needs ${range}. Run: introspection upgrade`);
+  }
+}
+
+// The Recipe's Python (package.json#pi.runtime.python): the documents tool runs its Word and PDF helpers with
+// VIRTUAL_ENV's python. Pi doesn't build that environment, so build it here from the committed lockfile, never
+// into system Python, and refuse to start without it. It goes under ~/.introspection, not into the package:
+// a global install can sit in a folder that needs sudo. Keyed by version and lockfile, so it's rebuilt only
+// when the pins change and versions sit side by side.
+const python = pkg.pi?.runtime?.python;
+if (python) {
+  if (!has("uv")) fail("uv is not installed; the documents tool needs it. Install it: https://docs.astral.sh/uv/getting-started/installation/");
+  const project = join(root, python.project);
+  const lock = readFileSync(join(root, python.lockfile));
+  const key = createHash("sha256").update(lock).digest("hex").slice(0, 12);
+  const venv = join(homedir(), ".introspection", "operator-python", `${pkg.version}-${key}`);
+  const uvEnv = { ...env, UV_PROJECT_ENVIRONMENT: venv };
+  const sync = spawnSync("uv", ["sync", "--frozen", "--no-install-project", "--quiet", "--project", project], { env: uvEnv, stdio: "inherit", shell: process.platform === "win32" });
+  if (sync.status !== 0) fail(`couldn't build the Recipe's Python environment from ${join(root, python.lockfile)}`);
+  const bin = join(venv, process.platform === "win32" ? "Scripts" : "bin");
+  env.VIRTUAL_ENV = venv;
+  env.PATH = [bin, env.PATH].join(delimiter);
+  delete env.PYTHONHOME;
+  const imports = python.imports ?? [];
+  if (imports.length > 0) {
+    const check = spawnSync(join(bin, "python"), ["-c", imports.map((name) => `import ${name}`).join("; ")], { env, stdio: "inherit" });
+    if (check.status !== 0) fail(`the Recipe's Python environment is missing one of: ${imports.join(", ")}`);
+  }
+}
+
+// The folder: the first argument when it isn't a flag, else the next new-<n>. A relative folder resolves
+// from where the command was typed (pnpm runs scripts from the package root, so INIT_CWD first).
+const typedFrom = process.env.INIT_CWD || process.cwd();
+let target;
+if (args.length > 0 && !args[0].startsWith("-")) {
+  target = resolve(typedFrom, args.shift());
+} else {
+  let n = 1;
+  while (existsSync(join(agentsDir, `new-${n}`))) n++;
+  target = join(agentsDir, `new-${n}`);
+}
+
+// Check the folder before creating anything, from its nearest existing parent.
+let existing = target;
+while (!existsSync(existing)) existing = dirname(existing);
+existing = realpathSync(existing);
+const inside = (parent, child) => {
+  const rel = relative(parent, child);
+  return rel === "" || (!rel.startsWith("..") && !isAbsolute(rel));
+};
+if (inside(realpathSync(root), existing)) fail(`${target} is inside the Operator package; choose a folder outside it`);
+// A folder inside another repository would nest the agent's repository there. The agent's own repository
+// is fine: that's how a build resumes.
+let top;
+try {
+  top = execFileSync("git", ["-C", existing, "rev-parse", "--show-toplevel"], { encoding: "utf8", stdio: ["ignore", "pipe", "ignore"] }).trim();
+} catch {}
+if (top) {
+  top = realpathSync(top);
+  if (!existsSync(target) || realpathSync(target) !== top) {
+    fail(`${target} is inside the git repository ${top}; choose a folder outside it, or the agent's own repository to resume`);
+  }
+}
+
+mkdirSync(target, { recursive: true });
+target = realpathSync(target);
+// The local equivalent of the cloud's workspace: files/ for the documents the owner shares (the documents tool
+// opens files only inside the build folder) and outputs/ for what the agent produces, both beside the agent's
+// repository, not in it. local-workspace.mjs tells the agent where they are.
+for (const dir of ["files", "outputs"]) mkdirSync(join(target, dir), { recursive: true });
+console.error(`introspection-operator: put the documents you'll share in ${join(target, "files")}`);
+console.error(`introspection-operator: starting in ${target}`);
+// Pi would load AGENTS.md and CLAUDE.md from this folder and every folder above it. Those are written for
+// coding agents and can describe other agents, which build must not copy; the Recipe brings its own.
+const result = spawnSync("pi", ["--recipe", root, "--agent", "build", "--no-context-files", "-e", join(root, "bin", "local-workspace.mjs"), ...args], {
+  cwd: target,
+  env,
+  stdio: "inherit",
+  shell: process.platform === "win32",
+});
+process.exit(result.status ?? 1);

package/bin/local-workspace.mjs

@@ -0,0 +1,62 @@
+// A Pi extension the launcher loads (`pi -e`) for a local build. It tells the build agent where the
+// workspace's folders are, as the cloud host does for every session (introspection-cloud
+// apps/runtime-agent/runtime/resources/context.ts, "Managed workspace"), so the Recipe itself says the same
+// thing locally and in the cloud and holds no local paths.
+//
+// It adds a message to the conversation rather than changing the system prompt: with a root SYSTEM.md, the
+// Recipes extension replaces Pi's whole prompt, so appended text (`--append-system-prompt`, or a
+// before_agent_start handler that runs first) never reaches the model. The message is added once per session,
+// so continuing a session with `-c` doesn't repeat it.
+import { delimiter, join } from "node:path";
+
+const TYPE = "operator-workspace";
+
+// Commands the agent runs must not inherit the build agent's own launch. The Recipes extension exports
+// PI_AGENT_NAME (`build`) for shell tools, and a Recipe launch reads it as its default agent, so
+// `introspection local` in the agent's repository looks for an agent named `build` there and doesn't start.
+// So every bash command starts by unsetting it. Windows shells are left as they are.
+//
+// The same goes for the Operator's own Python: the launcher starts Pi with it (VIRTUAL_ENV, its bin first on
+// PATH) for the Operator's documents tool. A Recipe run with `introspection local` that inherits it uses the
+// Operator's environment, so it passes without one of its own and fails for its owner and in the cloud.
+const operatorPython = process.env.VIRTUAL_ENV;
+const UNSET = ["PI_AGENT_NAME", "VIRTUAL_ENV"];
+
+function withoutOperatorEnv(command) {
+  if (process.platform === "win32") return command;
+  const unset = `unset ${UNSET.join(" ")}`;
+  if (!operatorPython) return `${unset}\n${command}`;
+  const bin = join(operatorPython, "bin");
+  const path = (process.env.PATH ?? "").split(delimiter).filter((dir) => dir !== bin).join(delimiter);
+  return `${unset}; export PATH='${path.replaceAll("'", `'\\''`)}'\n${command}`;
+}
+
+export default function localWorkspace(pi) {
+  pi.on("tool_call", (event) => {
+    if (event.toolName === "bash" && typeof event.input?.command === "string") {
+      event.input.command = withoutOperatorEnv(event.input.command);
+    }
+  });
+  pi.on("before_agent_start", (_event, ctx) => {
+    if (ctx.sessionManager.getEntries().some((entry) => entry.type === "custom_message" && entry.customType === TYPE)) return;
+    const root = ctx.cwd ?? process.cwd();
+    const files = join(root, "files");
+    const outputs = join(root, "outputs");
+    const content = [
+      "## Managed workspace",
+      "",
+      "<workspace>",
+      `  <inputs path="${files}" access="read_write" persistence="local" />`,
+      `  <outputs path="${outputs}" persistence="local" />`,
+      "  Create the agent's repository in this folder, beside files/ and outputs/.",
+      "</workspace>",
+      `This build runs on the owner's computer, and the documents tool opens files only inside ${root}. ` +
+        `The owner shares a document by giving you its path, or by putting it in ${files}: ask for it that way ` +
+        `("send me the path to it, or put it in ${files}"), not as an upload. ` +
+        "When the owner gives you a path to a file outside files/, copy it into files/ first and read the copy; " +
+        "don't try the original path, which the tool refuses. Never copy a file straight into the agent's repository. " +
+        "What belongs in the Recipe is added from files/.",
+    ].join("\n");
+    return { message: { customType: TYPE, content, display: false } };
+  });
+}

package/extensions/case-runs/events.mjs

@@ -0,0 +1,223 @@
+// Reads a case run's Pi event stream (`--mode json`) as it arrives and says what the run is doing: finished,
+// didn't start, failed, working (and on what), a tool that hasn't returned, a model that went quiet, or a loop.
+// Pi's events carry no times, so the reader stamps each one when it arrives.
+
+export const LIMITS = {
+  // A tool that hasn't returned in this long is hung.
+  toolHungMs: 15 * 60_000,
+  // No event at all for this long, outside a tool, means the model stalled.
+  stalledMs: 10 * 60_000,
+  // The same call with the same arguments this often, or the same error this often, is a loop.
+  sameCall: 4,
+  sameError: 3,
+};
+
+export function createRunReader() {
+  const state = {
+    events: 0,
+    lastEventAt: undefined,
+    turns: 0,
+    outputTokens: 0,
+    message: undefined, // the assistant message being generated: { kind, since, chars, tokens }
+    tools: new Map(), // running tools by call id: { name, args, since }
+    toolCounts: {},
+    calls: new Map(), // "name args" -> count
+    errors: [], // { tool, text }
+    errorCounts: new Map(),
+    retries: [],
+    finalText: undefined,
+    modelError: undefined,
+    ended: false,
+  };
+  let partial = "";
+
+  function onEvent(event, now) {
+    state.events++;
+    state.lastEventAt = now;
+    switch (event.type) {
+      case "turn_start":
+        state.turns++;
+        break;
+      case "message_start":
+        if (event.message?.role === "assistant") state.message = { kind: "waiting for the model", since: now, chars: 0, tokens: 0 };
+        break;
+      case "message_update": {
+        const delta = event.assistantMessageEvent;
+        const kind = { thinking_delta: "thinking", text_delta: "writing its reply", toolcall_delta: "writing a tool call" }[delta?.type];
+        if (!state.message) state.message = { kind: "waiting for the model", since: now, chars: 0, tokens: 0 };
+        if (kind && kind !== state.message.kind) Object.assign(state.message, { kind, since: now, chars: 0 });
+        if (typeof delta?.delta === "string") state.message.chars += delta.delta.length;
+        if (typeof event.usage?.output === "number") state.message.tokens = event.usage.output;
+        break;
+      }
+      case "message_end": {
+        const message = event.message;
+        if (message?.role !== "assistant") break;
+        state.outputTokens += message.usage?.output ?? state.message?.tokens ?? 0;
+        state.message = undefined;
+        const text = textOf(message.content);
+        if (text) state.finalText = text;
+        if (message.stopReason === "error" || message.stopReason === "aborted") {
+          state.modelError = message.errorMessage ?? message.stopReason;
+        } else if (message.stopReason) {
+          state.modelError = undefined;
+        }
+        break;
+      }
+      case "tool_execution_start":
+        state.tools.set(event.toolCallId, { name: event.toolName, args: event.args, since: now });
+        state.toolCounts[event.toolName] = (state.toolCounts[event.toolName] ?? 0) + 1;
+        {
+          const key = `${event.toolName} ${JSON.stringify(event.args ?? {})}`;
+          state.calls.set(key, (state.calls.get(key) ?? 0) + 1);
+        }
+        break;
+      case "tool_execution_end":
+        state.tools.delete(event.toolCallId);
+        if (event.isError) {
+          const text = firstLine(textOf(event.result?.content));
+          state.errors.push({ tool: event.toolName, text });
+          const key = `${event.toolName}: ${text}`;
+          state.errorCounts.set(key, (state.errorCounts.get(key) ?? 0) + 1);
+        }
+        break;
+      case "auto_retry_start":
+        state.retries.push(event.errorMessage ?? `attempt ${event.attempt}`);
+        break;
+      case "agent_end":
+        state.ended = true;
+        break;
+    }
+  }
+
+  return {
+    state,
+    // Feed raw stdout; returns the complete lines, for writing to events.jsonl unchanged.
+    feed(chunk, now = Date.now()) {
+      partial += chunk;
+      const lines = partial.split("\n");
+      partial = lines.pop();
+      for (const line of lines) {
+        if (!line.trim()) continue;
+        try {
+          onEvent(JSON.parse(line), now);
+        } catch {
+          // Not an event (a warning printed to stdout): keep it in the file, ignore it here.
+        }
+      }
+      return lines;
+    },
+  };
+}
+
+// The run's state in one verdict and a few lines of evidence.
+// run: { started, budgetMs, exit: { code, signal } | undefined, stopReason, stderr, outputs: [{ path, bytes }] }
+export function describe(state, run, now = Date.now()) {
+  const lines = [];
+  const elapsed = now - run.started;
+  const tools = Object.entries(state.toolCounts).map(([name, n]) => `${name} ${n}`).join(", ") || "none";
+  const progress = `${minutes(elapsed)}, turn ${state.turns}, ${state.outputTokens + (state.message?.tokens ?? 0)} output tokens, tool calls: ${tools}`;
+  const loop = findLoop(state);
+  let verdict;
+  let problem;
+
+  if (run.exit) {
+    if (run.stopReason) {
+      verdict = `stopped: ${run.stopReason}`;
+      problem = run.stopProblem;
+    } else if (run.exit.code !== 0 && state.turns === 0) {
+      verdict = "didn't start";
+      problem = "start";
+    } else if (run.exit.code !== 0 || state.modelError) {
+      verdict = `failed${state.modelError ? `: the model returned an error (${firstLine(state.modelError)})` : ` with exit code ${run.exit.code ?? run.exit.signal}`}`;
+      problem = "failed";
+    } else if (run.outputs.length === 0 && !state.finalText) {
+      verdict = "finished with no output: no reply, and outputs/ is empty";
+      problem = "no-output";
+    } else if (run.outputs.length === 0) {
+      verdict = "finished: replied, and wrote no files to outputs/";
+    } else {
+      verdict = "finished";
+    }
+  } else {
+    const hung = longestTool(state, now);
+    if (hung && now - hung.since >= LIMITS.toolHungMs) {
+      verdict = `tool hung: ${hung.name} has run ${minutes(now - hung.since)} (${summarize(hung.args)})`;
+      problem = "tool-hung";
+    } else if (!hung && state.lastEventAt !== undefined && now - state.lastEventAt >= LIMITS.stalledMs) {
+      verdict = `model stalled: no event for ${minutes(now - state.lastEventAt)}`;
+      problem = "stalled";
+    } else if (state.lastEventAt === undefined && elapsed >= LIMITS.stalledMs) {
+      verdict = `no events after ${minutes(elapsed)}: the run hasn't started a session`;
+      problem = "stalled";
+    } else if (run.budgetMs && elapsed >= run.budgetMs) {
+      verdict = `over budget while working (${minutes(run.budgetMs)})`;
+      problem = "over-budget";
+    } else if (loop) {
+      verdict = `looping: ${loop}`;
+      problem = "looping";
+    } else if (hung) {
+      verdict = `working: running ${hung.name} for ${minutes(now - hung.since)} (${summarize(hung.args)})`;
+    } else if (state.message) {
+      verdict = `working: ${state.message.kind} for ${minutes(now - state.message.since)}${state.message.chars ? ` (${state.message.chars} characters so far)` : ""}`;
+    } else {
+      verdict = "working";
+    }
+  }
+
+  lines.push(verdict);
+  if (state.modelError) {
+    lines.push("this is the model provider, not the agent: fix access, credentials or quota and run it again; the agent keeps the model it was designed with");
+  }
+  lines.push(`progress: ${progress}`);
+  if (loop && problem !== "looping") lines.push(`repeating: ${loop}`);
+  if (state.retries.length) lines.push(`model retries: ${state.retries.length} (last: ${firstLine(state.retries.at(-1))})`);
+  if (state.errors.length) {
+    lines.push(`tool errors: ${state.errors.length}`);
+    for (const error of state.errors.slice(-5)) lines.push(`  ${error.tool}: ${error.text}`);
+  }
+  if (run.exit) {
+    lines.push(`outputs: ${run.outputs.length ? run.outputs.map((o) => `${o.path} (${o.bytes} bytes)`).join(", ") : "none"}`);
+    if (state.finalText) lines.push(`final reply: ${truncate(state.finalText, 1200)}`);
+    if (run.exit.code !== 0 && run.stderr) lines.push(`stderr (last lines):\n${tail(run.stderr, 15)}`);
+  }
+  return { verdict, problem, text: lines.join("\n") };
+}
+
+function longestTool(state, now) {
+  let longest;
+  for (const tool of state.tools.values()) if (!longest || tool.since < longest.since) longest = tool;
+  return longest && now >= longest.since ? longest : undefined;
+}
+
+function findLoop(state) {
+  for (const [key, n] of state.calls) if (n >= LIMITS.sameCall) return `the same call ${n} times: ${truncate(key, 200)}`;
+  for (const [key, n] of state.errorCounts) if (n >= LIMITS.sameError) return `the same error ${n} times: ${truncate(key, 200)}`;
+  return undefined;
+}
+
+function textOf(content) {
+  if (typeof content === "string") return content;
+  return (Array.isArray(content) ? content : []).filter((c) => c?.type === "text").map((c) => c.text).join("\n").trim();
+}
+
+function summarize(args) {
+  return truncate(JSON.stringify(args ?? {}), 160);
+}
+
+function firstLine(text) {
+  return truncate(String(text ?? "").trim().split("\n")[0], 200);
+}
+
+function truncate(text, max) {
+  return text.length > max ? `${text.slice(0, max)}…` : text;
+}
+
+function tail(text, n) {
+  return text.trimEnd().split("\n").slice(-n).join("\n");
+}
+
+export function minutes(ms) {
+  const s = Math.round(ms / 1000);
+  return s < 90 ? `${s}s` : `${Math.floor(s / 60)} min ${s % 60}s`;
+}

package/extensions/case-runs/index.mjs

@@ -0,0 +1,88 @@
+// Tools the build agent runs an agent's cases with: run_case starts one run of the Recipe being built in the
+// background, case_status says what it's doing, stop_case ends it. The runs themselves are in runs.mjs, and
+// what their events mean in events.mjs.
+import { relative } from "node:path";
+import { Type } from "typebox";
+import { minutes } from "./events.mjs";
+import { createCaseRuns, DEFAULT_BUDGET_MINUTES, MAX_WAIT_SECONDS } from "./runs.mjs";
+
+function text(value) {
+  return { content: [{ type: "text", text: value }], details: {} };
+}
+
+export default function caseRuns(pi) {
+  const caseRuns = createCaseRuns();
+
+  function find(id) {
+    const run = caseRuns.runs.get(id);
+    if (!run) throw new Error(`no case run ${id} in this session; runs started: ${[...caseRuns.runs.keys()].join(", ") || "none"}`);
+    return run;
+  }
+
+  pi.registerTool({
+    name: "run_case",
+    label: "Run case",
+    description:
+      "Run the agent being built on one case, the way a real request runs, in the background. The run gets its own folder runs/<case>-<n>/ in the workspace, with the case's input files in its files/ and an empty outputs/, and starts `introspection local --work-dir <repository> -p <prompt>` there. Its session and event stream (events.jsonl) stay in that folder. Returns the run's id at once; follow it with case_status.",
+    parameters: Type.Object({
+      case: Type.String({ description: "The case's id, such as C1; names the run's folder." }),
+      prompt: Type.String({ description: "The request, holding only what a real request brings." }),
+      repository: Type.String({ description: "The agent's repository, relative to the workspace." }),
+      files: Type.Optional(Type.Array(Type.String(), { description: "Input files for the run's files/, relative to the workspace or absolute." })),
+      agent: Type.Optional(Type.String({ description: "The Recipe agent to run, when it isn't the default." })),
+      budget_minutes: Type.Optional(Type.Number({ description: `Stop the run after this many minutes even while it's working (default ${DEFAULT_BUDGET_MINUTES}).` })),
+      env: Type.Optional(Type.Record(Type.String(), Type.String(), { description: "Environment the agent's own runtime needs, such as its Python environment's VIRTUAL_ENV." })),
+    }),
+    async execute(_id, params, _signal, _onUpdate, ctx) {
+      const cwd = ctx?.cwd ?? process.cwd();
+      const run = caseRuns.start({
+        cwd,
+        caseId: params.case,
+        prompt: params.prompt,
+        repository: params.repository,
+        files: params.files,
+        agent: params.agent,
+        budgetMinutes: params.budget_minutes,
+        env: params.env,
+      });
+      return text(`Started ${run.id} in ${relative(cwd, run.dir)}/ (budget ${minutes(run.budgetMs)}). Check it with case_status ${run.id}; pass wait_seconds to wait for it.`);
+    },
+  });
+
+  pi.registerTool({
+    name: "case_status",
+    label: "Case status",
+    description:
+      "What a case run is doing, from its events: finished (with its outputs and final reply), didn't start, failed, working (thinking, writing, or which tool for how long), tool hung, model stalled, looping, or stopped and why. With wait_seconds it returns when the run ends, when a problem shows, or when the wait is over.",
+    parameters: Type.Object({
+      id: Type.String({ description: "The run's id from run_case, such as C1-1." }),
+      wait_seconds: Type.Optional(Type.Number({ description: `Wait up to this long (at most ${MAX_WAIT_SECONDS}).` })),
+    }),
+    async execute(_id, params, signal) {
+      const run = find(params.id);
+      if (params.wait_seconds) await caseRuns.wait(run, params.wait_seconds, signal);
+      return text(`${run.id}: ${caseRuns.status(run).text}`);
+    },
+  });
+
+  pi.registerTool({
+    name: "stop_case",
+    label: "Stop case",
+    description: "Stop a case run that is still going, and say what it was doing when stopped.",
+    parameters: Type.Object({
+      id: Type.String({ description: "The run's id from run_case." }),
+      reason: Type.Optional(Type.String({ description: "Why it's stopped, recorded in its run.json." })),
+    }),
+    async execute(_id, params) {
+      const run = find(params.id);
+      const before = caseRuns.status(run).verdict;
+      caseRuns.stop(run, params.reason ?? `stopped by the build agent while ${before}`, "stopped");
+      await caseRuns.wait(run, 15);
+      return text(`${run.id}: ${caseRuns.status(run).text}`);
+    },
+  });
+
+  pi.on("session_shutdown", async () => {
+    await caseRuns.stopAll();
+  });
+}