@pify/shell-background 0.1.2 → 0.3.0

package/README.md

@@ -12,7 +12,7 @@ pi's bash tool waits for the command to finish. That is right for `ls` and wrong
 
 ## What it does
 
-It re-registers the `bash` tool with the same shell, working directory and environment — nothing about how a command runs changes — but a different lifecycle:
+It re-registers the `bash` tool with the same shell, working directory, PATH (including pi's managed `fd`/`rg` bin dir) and `PI_*` session variables pi's own bash hands a command — but a different lifecycle:
 
 | Situation | What happens |
 |---|---|
@@ -25,6 +25,7 @@ It re-registers the `bash` tool with the same shell, working directory and envir
 bash { command: "npm run build" }        # returns when done, or auto-backgrounds at 30s
 bash { command: "npm run dev", background: true }   # → "bg-2 started in the background"
 shell_status { id: "bg-2" }              # status + output so far
+shell_status { id: "bg-2", wait: 60 }    # block up to 60s until it finishes (headless collect)
 shell_status                             # list every background command this session
 shell_kill { id: "bg-2" }                # stop it and its whole process tree
 ```
@@ -33,13 +34,15 @@ shell_kill { id: "bg-2" }                # stop it and its whole process tree
 
 ## Delivery, and the headless caveat
 
-When a backgrounded command finishes in an **interactive** session, its result is pushed into the conversation as the next turn — you do not have to poll. Under headless `pi -p` there is nothing to deliver into (the session tears down when the prompt resolves), so **auto-background is disabled there** and only explicit `background: true` applies; collect it with `shell_status` inside the same turn. This is the same delivery rule the rest of the suite lives by.
+When a backgrounded command finishes in an **interactive** session, its result is pushed into the conversation as the next turn — you do not have to poll. Under headless `pi -p` there is nothing to deliver into (the session tears down when the prompt resolves), so **auto-background is disabled there** and only explicit `background: true` applies; collect it within the same turn with `shell_status { id, wait: N }`, which blocks (up to `N` seconds, 0–300) until the command finishes rather than returning immediately. This is the same delivery rule the rest of the suite lives by.
 
 ## How it works
 
 Each command is spawned with its stdout and stderr piped into a single log file, drained on every chunk so nothing is lost no matter how much it prints, and finalized only after the pipes end (with a short grace so a daemonized grandchild that holds a handle open cannot truncate the tail). The process is spawned detached (POSIX) and `unref`'d so a running job never holds the host open, and killed as a whole process tree — `taskkill /T` on Windows, a process-group signal on POSIX — on timeout, abort, `shell_kill`, or session shutdown.
 
-Shell resolution reuses pi's own `getShellConfig` (Git Bash on Windows, `/bin/bash` then `sh` on Unix), so a backgrounded command behaves identically to a foreground one. Jobs are tracked in memory and mirrored to a per-session sidecar under the temp dir, so `shell_status` still answers after a `/reload` and a job whose process has died is reconciled rather than shown as forever-running.
+Shell resolution reuses pi's own `getShellConfig` (Git Bash on Windows, `/bin/bash` then `sh` on Unix) and the environment is rebuilt the way pi's bash builds it (managed bin dir on PATH, `PI_SESSION_ID`/`PI_SESSION_FILE`/`PI_PROVIDER`/`PI_MODEL`/`PI_REASONING_LEVEL` from the session), so a backgrounded command behaves identically to a foreground one.
+
+Jobs are tracked in memory and mirrored to a sidecar under the temp dir, keyed by the pi **session id** — so two sessions in the same directory never see or kill each other's jobs, and the id is stable across a `/reload`. A `/reload` does **not** kill background jobs: pi hands the same host process to a fresh instance, which adopts every still-running job from the sidecars and delivers each one when it finishes (exactly once). Every other way a session ends — quit, or switching to another session — kills its jobs and their whole process trees. Each record also carries the host pid that spawned it: a `running` record left by a different (or crashed) host is surfaced as `orphaned` and never treated as live, so its pid — which may since belong to something unrelated — is never signalled. A job whose process has died is reconciled rather than shown as forever-running, this session's dir is removed on a clean exit, and stray dirs from a crash are swept after seven days.
 
 There are **no runtime dependencies**, and it works on Linux, macOS and Windows.
 
@@ -50,11 +53,14 @@ Put these in `.pi/shell-background.json` (project) or `<agentDir>/shell-backgrou
 ```json
 {
   "autoBackgroundMs": 30000,
-  "tailBytes": 65536
+  "tailBytes": 65536,
+  "maxBackground": 8
 }
 ```
 
-`autoBackgroundMs` is how long a foreground command may run before it auto-backgrounds; set it to `0` to disable auto-background (explicit `background: true` still works). `PIFY_SHELL_BG_MS` overrides it for one run or in CI. `tailBytes` bounds how much of a job's log a status result shows. Bad values fall back to the defaults with a warning rather than taking the tool down.
+`autoBackgroundMs` is how long a foreground command may run before it auto-backgrounds; set it to `0` to disable auto-background (explicit `background: true` still works). `PIFY_SHELL_BG_MS` overrides it for one run or in CI. `tailBytes` bounds how much of a job's log a status result shows. `maxBackground` bounds how many jobs may be alive at once: over it, a `background: true` request is refused with a message naming the limit (the command can still run in the foreground), and a command that crosses the auto-background threshold simply stays in the foreground instead of moving — a command is never refused, only the decision to background it. Bad values fall back to the defaults with a warning rather than taking the tool down.
+
+The tail a result carries is cleaned the way pi's own bash cleans what the model sees — ANSI escapes, control characters and carriage-return progress frames stripped — so a chatty build or dev server does not spend tokens on colour codes and thousands of overwritten progress lines. The log file on disk keeps every byte.
 
 ## Coexistence with @pify/pretty
 

package/extensions/shell-background.ts

@@ -37,15 +37,17 @@ import { Type } from "typebox";
 import { tmpdir } from "node:os";
 import { join } from "node:path";
 import { createHash } from "node:crypto";
-import { readFileSync } from "node:fs";
+import { readFileSync, readdirSync, statSync, rmSync } from "node:fs";
 
-import { JobRegistry } from "../src/registry.ts";
+import { JobRegistry, isAlive } from "../src/registry.ts";
 import { spawnToFile } from "../src/spawn.ts";
 import { killTree } from "../src/kill.ts";
 import { readTail } from "../src/tail.ts";
+import { buildEnv } from "../src/env.ts";
 import { DEFAULT_SETTINGS, resolveSettings, type ShellBgSettings } from "../src/config.ts";
 import { backgroundedResult, deliveryMessage, DELIVERY_TYPE } from "../src/pending.ts";
 import { formatResult, formatList, header } from "../src/format.ts";
+import { sanitizeOutput } from "../src/sanitize.ts";
 import { buildWidgetLines } from "../src/widget.ts";
 import { isFinished } from "../src/types.ts";
 import type { Job } from "../src/types.ts";
@@ -60,13 +62,67 @@ export default function shellBackground(pi: ExtensionAPI) {
   let settings: ShellBgSettings = DEFAULT_SETTINGS;
   let registry: JobRegistry | null = null;
   let lastUiCtx: UiContext | null = null;
+  let widgetTimer: NodeJS.Timeout | null = null;
+  // Set on a /reload shutdown: this (old) instance's processes are being handed
+  // to the next instance in the same host process, so its settle/delivery
+  // closures must go quiet — flush status to disk, but never kill and never
+  // deliver through the now-stale pi handle. Each instance has its own copy
+  // (reload builds a fresh closure), so this only ever flips once, on the way out.
+  let handedOff = false;
+
+  const ROOT = join(tmpdir(), "pify-shell-bg");
+  const MAX_SESSION_DIR_AGE_MS = 7 * 24 * 60 * 60 * 1000;
+
+  // Ref'd on purpose: this backs the shell_status `wait` poll, which is awaited
+  // inside an in-flight tool call, so the timer must actually resolve rather than
+  // let an otherwise-idle loop exit out from under it.
+  const sleep = (ms: number) => new Promise<void>((r) => setTimeout(r, ms));
+
+  /**
+   * The one auto-background threshold, computed once so the bash guideline and
+   * runBash cannot drift: auto-background needs a session that outlives the run
+   * to deliver into, which a headless `pi -p` does not have, so it is off there.
+   */
+  const effectiveAutoMs = (hasUI: boolean): number => (hasUI ? settings.autoBackgroundMs : 0);
 
   // ── setup ──────────────────────────────────────────────────────────
 
-  function sessionKey(cwd: string): string {
-    const id = process.env.PI_SESSION_ID;
+  /**
+   * A stable per-session key: the session id survives a /reload (so adopted
+   * jobs are found again) and differs across /new and /resume (so sessions never
+   * reconcile or kill each other's jobs). Falls back to a cwd hash only when no
+   * session manager is present (e.g. the unit harness).
+   */
+  function sessionKey(ctx: UiContext): string {
+    let id: string | undefined;
+    try {
+      id = ctx.sessionManager?.getSessionId?.();
+    } catch {
+      // ignore — fall through to the cwd hash
+    }
     if (id) return id.replace(/[^A-Za-z0-9_-]/g, "_").slice(0, 40);
-    return createHash("sha256").update(cwd).digest("hex").slice(0, 16);
+    return createHash("sha256").update(ctx.cwd).digest("hex").slice(0, 16);
+  }
+
+  /** Remove sibling session dirs untouched for a week (a crash leaves the dir). */
+  function sweepOldDirs(keep: string): void {
+    let names: string[];
+    try {
+      names = readdirSync(ROOT);
+    } catch {
+      return;
+    }
+    const cutoff = Date.now() - MAX_SESSION_DIR_AGE_MS;
+    for (const name of names) {
+      if (name === keep) continue; // never sweep the session we are starting
+      const p = join(ROOT, name);
+      try {
+        const st = statSync(p);
+        if (st.isDirectory() && st.mtimeMs < cutoff) rmSync(p, { recursive: true, force: true });
+      } catch {
+        // A dir we cannot stat or remove is not worth failing startup over.
+      }
+    }
   }
 
   function loadSettings(cwd: string): string[] {
@@ -100,6 +156,13 @@ export default function shellBackground(pi: ExtensionAPI) {
     return { shell: cfg.shell, args };
   }
 
+  function stopWidgetTimer(): void {
+    if (widgetTimer) {
+      clearInterval(widgetTimer);
+      widgetTimer = null;
+    }
+  }
+
   function renderWidget(ctx: UiContext | null = lastUiCtx): void {
     if (!ctx || !ctx.hasUI || !registry) return;
     lastUiCtx = ctx;
@@ -107,9 +170,17 @@ export default function shellBackground(pi: ExtensionAPI) {
     const lines = buildWidgetLines(registry.all(), ctx.ui.theme as never, now);
     if (lines.length === 0) {
       ctx.ui.setWidget(WIDGET, undefined);
+      stopWidgetTimer();
       return;
     }
     ctx.ui.setWidget(WIDGET, (_tui: unknown) => new Text(lines.join("\n"), 0, 0), { placement: "aboveEditor" });
+    // Keep the box live between events: tick the elapsed clock while jobs run and
+    // clear a finished job once it falls out of its 15s window, even if nothing
+    // else fires a render. Unref'd, so a lingering box never holds the host open.
+    if (!widgetTimer) {
+      widgetTimer = setInterval(() => renderWidget(), 1000);
+      widgetTimer.unref?.();
+    }
   }
 
   // ── run ────────────────────────────────────────────────────────────
@@ -117,7 +188,7 @@ export default function shellBackground(pi: ExtensionAPI) {
   function snapshot(job: Job): ToolResult {
     const tail = readTail(job.logPath, settings.tailBytes);
     return {
-      content: [{ type: "text", text: `${header(job)}\n${tail.text.replace(/\n+$/, "") || "(no output yet)"}` }],
+      content: [{ type: "text", text: `${header(job)}\n${sanitizeOutput(tail.text).replace(/\n+$/, "") || "(no output yet)"}` }],
       details: { id: job.id, status: job.status },
     };
   }
@@ -130,23 +201,36 @@ export default function shellBackground(pi: ExtensionAPI) {
     };
   }
 
+  /** Push a finished job's result into the conversation through the live pi. */
+  function deliver(job: Job): void {
+    renderWidget();
+    pi.sendMessage(
+      {
+        customType: DELIVERY_TYPE,
+        content: deliveryMessage(job.id, formatResult(job, settings.tailBytes)),
+        display: true,
+        details: { id: job.id, status: job.status, exitCode: job.exitCode },
+      },
+      { deliverAs: "followUp", triggerTurn: true },
+    );
+  }
+
   /** Deliver a finished background job into the conversation, once. */
   function scheduleDelivery(job: Job, exit: Promise<unknown>): void {
     exit
       .then(() => {
+        // Handed off to a newer instance: only flush the final status to disk
+        // for it to read. Do not mark delivered, touch lastUiCtx, or send through
+        // this stale pi (which would throw and, worse, having already flipped
+        // delivered would stop the new instance from ever delivering).
+        if (handedOff) {
+          registry?.persist(job);
+          return;
+        }
         if (job.delivered) return;
         job.delivered = true;
         registry?.persist(job);
-        renderWidget();
-        pi.sendMessage(
-          {
-            customType: DELIVERY_TYPE,
-            content: deliveryMessage(job.id, formatResult(job, settings.tailBytes)),
-            display: true,
-            details: { id: job.id, status: job.status, exitCode: job.exitCode },
-          },
-          { deliverAs: "followUp", triggerTurn: true },
-        );
+        deliver(job);
       })
       .catch(() => {
         // A /reload can make captured handles throw; delivery is a convenience,
@@ -165,12 +249,33 @@ export default function shellBackground(pi: ExtensionAPI) {
     const command = String(params.command ?? "").trim();
     if (!command) return { content: [{ type: "text", text: "Empty command." }], details: {}, isError: true };
 
+    // The cap bounds how many jobs may be alive, not whether a command runs:
+    // an explicit background request over it is refused (the model can still
+    // run the command in the foreground); an auto-background transition over
+    // it is skipped and the command simply stays in the foreground.
+    const liveOthers = (self?: string) => registry!.running().filter((j) => j.id !== self).length;
+    if (params.background && liveOthers() >= settings.maxBackground) {
+      const n = liveOthers();
+      return {
+        content: [
+          {
+            type: "text",
+            text:
+              `${n} background command${n === 1 ? " is" : "s are"} already running (maxBackground = ${settings.maxBackground}). ` +
+              `shell_kill one you no longer need, wait for one with shell_status {id, wait}, or run this in the foreground without background:true.`,
+          },
+        ],
+        details: { running: n, limit: settings.maxBackground },
+        isError: true,
+      };
+    }
+
     const job = registry.create(command, ctx.cwd);
     const { shell, args } = shellArgv();
 
     let spawned;
     try {
-      spawned = spawnToFile(shell, args, command, ctx.cwd, process.env, job.logPath);
+      spawned = spawnToFile(shell, args, command, ctx.cwd, buildEnv(ctx, join(getAgentDir(), "bin")), job.logPath);
     } catch (err) {
       job.status = "failed";
       job.endedAt = Date.now();
@@ -189,6 +294,12 @@ export default function shellBackground(pi: ExtensionAPI) {
       job.exitCode = code;
       job.signal = sig;
       job.endedAt = Date.now();
+      // Handed off on /reload: flush the final status to the sidecar for the new
+      // instance to read, but do not deliver or touch the stale UI ctx here.
+      if (handedOff) {
+        registry?.persist(job);
+        return;
+      }
       registry?.persist(job);
       renderWidget();
     });
@@ -208,7 +319,7 @@ export default function shellBackground(pi: ExtensionAPI) {
 
     // Foreground: race the process against the auto-background threshold, an
     // optional timeout, and the turn's abort signal — streaming the tail.
-    const autoMs = ctx.hasUI ? settings.autoBackgroundMs : 0;
+    const autoMs = effectiveAutoMs(ctx.hasUI);
     const timers: NodeJS.Timeout[] = [];
     const after = (ms: number, val: string) =>
       new Promise<string>((res) => {
@@ -227,13 +338,31 @@ export default function shellBackground(pi: ExtensionAPI) {
 
     try {
       const race: Array<Promise<string>> = [settle.then(() => "exit")];
-      if (autoMs > 0) race.push(after(autoMs, "auto"));
       if (params.timeout && params.timeout > 0) race.push(after(params.timeout * 1000, "timeout"));
       if (signal) race.push(abort);
 
-      const outcome = await Promise.race(race);
+      let outcome = await Promise.race(autoMs > 0 ? [...race, after(autoMs, "auto")] : race);
 
-      if (outcome === "exit") return finished(job);
+      // Over the cap, the threshold passes without moving the command: it is
+      // never refused, it just keeps its foreground slot until it ends.
+      let keptForeground = false;
+      if (outcome === "auto" && liveOthers(job.id) >= settings.maxBackground) {
+        keptForeground = true;
+        outcome = await Promise.race(race);
+      }
+
+      if (outcome === "exit") {
+        const r = finished(job);
+        if (keptForeground) {
+          r.content = [
+            {
+              type: "text",
+              text: `${r.content[0]?.text ?? ""}\n\n[kept in the foreground: ${settings.maxBackground} background commands were already running]`,
+            },
+          ];
+        }
+        return r;
+      }
 
       if (outcome === "auto") {
         scheduleDelivery(job, settle);
@@ -264,7 +393,27 @@ export default function shellBackground(pi: ExtensionAPI) {
       // timeout or abort: stop the tree, let the record settle, report partial.
       job.killedByUs = true;
       killTree(job.pid);
-      await Promise.race([settle, after(500, "gave-up")]);
+      // Give the tree time to actually die before reporting — up to killTree's
+      // own SIGKILL grace on a timeout, briefly on an abort so the turn ends.
+      const graceMs = outcome === "timeout" ? 3000 : 500;
+      await Promise.race([settle, after(graceMs, "gave-up")]);
+      if (outcome === "timeout" && job.status === "running") {
+        // Do not print "[killed]" under a still-"running" header: the signal is
+        // out but the process has not confirmed exit. Say so, and point at the
+        // status tool rather than claim a death we cannot see yet.
+        return {
+          content: [
+            {
+              type: "text",
+              text:
+                formatResult(job, settings.tailBytes) +
+                `\n\n[kill signal sent; process had not exited after 3s — shell_status ${job.id} to confirm]`,
+            },
+          ],
+          details: { id: job.id, status: job.status },
+          isError: true,
+        };
+      }
       const note =
         outcome === "timeout"
           ? `\n\n[killed: exceeded the ${params.timeout}s timeout]`
@@ -282,16 +431,31 @@ export default function shellBackground(pi: ExtensionAPI) {
 
   // ── tools & lifecycle ────────────────────────────────────────────────
 
-  function registerBash(cwd: string): void {
+  function registerBash(cwd: string, hasUI: boolean): void {
     const original = createBashToolDefinition(cwd) as unknown as AnyTool;
+    const autoMs = effectiveAutoMs(hasUI);
+    // The guideline must match what runBash will actually do for this session:
+    // auto-background only happens interactively with a positive threshold.
+    const backgroundLine = !hasUI
+      ? `This is a headless run: commands run to completion unless you pass background:true, which returns an id immediately. Nothing is delivered after your turn, so collect a backgrounded command within the same turn with shell_status {id, wait: N} (it blocks until the command finishes or the wait elapses).`
+      : autoMs === 0
+        ? `Commands run to completion; pass background:true for anything long-running (a server, build, or watcher) to get an id back immediately, then collect or check it with shell_status.`
+        : `A command still running after ${Math.round(autoMs / 1000)}s is moved to the background and its result is delivered when it finishes; pass background:true to background a long task (a server, build, or watcher) immediately. Collect or check with shell_status.`;
     const guidelines = [
       ...(Array.isArray((original as { promptGuidelines?: unknown }).promptGuidelines)
         ? ((original as { promptGuidelines?: string[] }).promptGuidelines as string[])
         : []),
-      `A command still running after ${Math.round(settings.autoBackgroundMs / 1000)}s is moved to the background and its result is delivered when it finishes; pass background:true to background a long task (a server, build, or watcher) immediately. Collect or check with shell_status.`,
+      backgroundLine,
     ];
     pi.registerTool({
       ...original,
+      // pi's description promises its own truncation ("last 2000 lines or
+      // 50KB … saved to a temp file"); this tool returns the last
+      // `tailBytes` of a log it keeps for the whole command, and says where.
+      description:
+        `Execute a shell command in the current working directory. Returns stdout and stderr, interleaved as they ` +
+        `arrived; a long output is shown as its last ${Math.round(settings.tailBytes / 1024)}KB with the path of the ` +
+        `full log. Optionally provide a timeout in seconds, or background:true to get an id back immediately.`,
       parameters: Type.Object({
         command: Type.String({ description: "Shell command to execute" }),
         timeout: Type.Optional(
@@ -315,11 +479,14 @@ export default function shellBackground(pi: ExtensionAPI) {
     label: "Background shell status",
     promptSnippet: "Check or collect a backgrounded command",
     description:
-      "Report a background command by id (its status and output tail), or list all this session's background commands when given no id. Finished results survive until the session ends.",
+      "Report a background command by id (its status and output tail), or list all this session's background commands when given no id. Pass wait (seconds, 0–300) to block until a still-running command finishes or the wait elapses — useful in a headless run where nothing is delivered after the turn. Finished results survive until the session ends.",
     parameters: Type.Object({
       id: Type.Optional(Type.String({ description: "Job id, e.g. bg-1. Omit to list all." })),
+      wait: Type.Optional(
+        Type.Number({ description: "Seconds to block while the command is still running (clamped 0–300). Default 0 — return immediately." }),
+      ),
     }),
-    async execute(_id: string, params: { id?: string }): Promise<ToolResult> {
+    async execute(_id: string, params: { id?: string; wait?: number }, sig?: AbortSignal): Promise<ToolResult> {
       if (!registry) return { content: [{ type: "text", text: "shell-background not initialized" }], details: {}, isError: true };
       const id = params.id?.trim();
       if (!id) return { content: [{ type: "text", text: formatList(registry.all()) }], details: {} };
@@ -328,6 +495,15 @@ export default function shellBackground(pi: ExtensionAPI) {
         const known = registry.all().map((j) => j.id).join(", ") || "(none)";
         return { content: [{ type: "text", text: `No job "${id}". Known: ${known}` }], details: {}, isError: true };
       }
+      // Optionally block until it finishes. A bounded poll of job.status (not an
+      // in-memory promise) so it works for adopted jobs too — those have no
+      // settle closure; their status is flipped by the adoption poller / a
+      // sibling instance persisting the sidecar. Cheap: status mutates in place.
+      const waitSec = Number.isFinite(params.wait) ? Math.min(300, Math.max(0, Math.floor(params.wait as number))) : 0;
+      if (waitSec > 0 && job.status === "running") {
+        const deadline = Date.now() + waitSec * 1000;
+        while (job.status === "running" && Date.now() < deadline && !sig?.aborted) await sleep(250);
+      }
       // Reading a finished job marks it collected so it will not also be
       // delivered unasked. A still-running poll must never do this: setting
       // delivered here would permanently cancel the promised auto-delivery.
@@ -381,19 +557,96 @@ export default function shellBackground(pi: ExtensionAPI) {
     },
   });
 
+  /**
+   * After load(), take over every job still running from a previous instance of
+   * this same host — a /reload survivor (a foreign/dead host's jobs were settled
+   * to orphaned by load() and are not here). Their settle/delivery closures went
+   * with the old instance, so poll the sidecar the old closure keeps flushing:
+   * when it flips to a finished status (or the pid is simply gone) copy that in,
+   * persist, re-render, and deliver through the live pi if it was not delivered.
+   * One unref'd interval for them all; it stops once none are still running.
+   */
+  function adoptRunning(ctx: UiContext): void {
+    if (!registry) return;
+    const adopted = registry.running();
+    if (adopted.length === 0) return;
+    lastUiCtx = ctx;
+    const timer = setInterval(() => {
+      if (!registry) {
+        clearInterval(timer);
+        return;
+      }
+      let anyRunning = false;
+      for (const job of adopted) {
+        if (job.status === "running") {
+          const disk = registry.readSidecar(job.id);
+          if (disk && disk.status !== "running") {
+            job.status = disk.status;
+            job.exitCode = disk.exitCode;
+            job.signal = disk.signal;
+            job.endedAt = disk.endedAt ?? Date.now();
+          } else if (!isAlive(job.pid)) {
+            // The old instance never flushed a final status (it crashed) but the
+            // process is gone: settle it here so it is not shown running forever.
+            job.status = job.killedByUs ? "killed" : "done";
+            job.endedAt = job.endedAt ?? Date.now();
+          }
+        }
+        if (job.status === "running") anyRunning = true;
+        else if (!job.delivered) {
+          job.delivered = true;
+          registry.persist(job);
+          deliver(job);
+        }
+      }
+      renderWidget(ctx);
+      if (!anyRunning) clearInterval(timer);
+    }, 1000);
+    timer.unref?.();
+  }
+
   pi.on("session_start", async (_event, ctx) => {
     const warnings = loadSettings(ctx.cwd);
-    registry = new JobRegistry(join(tmpdir(), "pify-shell-bg", sessionKey(ctx.cwd)));
+    const key = sessionKey(ctx);
+    sweepOldDirs(key);
+    registry = new JobRegistry(join(ROOT, key));
     registry.load();
-    registerBash(ctx.cwd);
+    registerBash(ctx.cwd, ctx.hasUI);
     renderWidget(ctx);
+    adoptRunning(ctx);
     if (warnings.length > 0 && ctx.hasUI) ctx.ui.notify(`shell-background settings: ${warnings.join("; ")}`, "warning");
   });
 
-  pi.on("session_shutdown", async (_event, ctx) => {
-    // Background jobs are tied to the session; do not leave orphans running
-    // after pi exits.
-    if (registry) for (const job of registry.running()) killTree(job.pid);
+  pi.on("session_shutdown", async (event, ctx) => {
+    if (event.reason === "reload") {
+      // Same host process, a fresh instance is coming right after: hand the jobs
+      // off rather than kill them. This (old) instance's closures go quiet
+      // (handedOff) and the new instance adopts the still-running ones from the
+      // sidecars — that is what registry.ts and the README promise across /reload.
+      handedOff = true;
+      stopWidgetTimer();
+      if (ctx.hasUI) ctx.ui.setWidget(WIDGET, undefined);
+      return;
+    }
+    // quit / new / resume / fork: this session is ending for good. Kill each
+    // running tree and record it killed synchronously so a resumed/next reader
+    // never sees a dead job as running, then best-effort drop this session's dir.
+    if (registry) {
+      for (const job of registry.running()) {
+        job.killedByUs = true;
+        killTree(job.pid);
+        job.status = "killed";
+        job.endedAt = job.endedAt ?? Date.now();
+        registry.persist(job);
+      }
+      try {
+        rmSync(registry.dir(), { recursive: true, force: true });
+      } catch {
+        // Windows can hold the log write stream open (EBUSY); the 7-day age
+        // sweep on a later session_start collects whatever is left behind.
+      }
+    }
+    stopWidgetTimer();
     if (ctx.hasUI) ctx.ui.setWidget(WIDGET, undefined);
   });
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@pify/shell-background",
-  "version": "0.1.2",
+  "version": "0.3.0",
   "description": "Long-running bash goes async: background: true launches detached, and any command still running after 30s auto-backgrounds and delivers its result when it finishes",
   "keywords": [
     "pi-package",
@@ -63,9 +63,9 @@
     }
   },
   "devDependencies": {
-    "@earendil-works/pi-ai": "^0.85.1",
-    "@earendil-works/pi-coding-agent": "^0.85.1",
-    "@earendil-works/pi-tui": "^0.85.1",
+    "@earendil-works/pi-ai": "^0.87.0",
+    "@earendil-works/pi-coding-agent": "^0.87.0",
+    "@earendil-works/pi-tui": "^0.87.0",
     "@types/node": "^22.10.2",
     "typebox": "^1.1.38",
     "typescript": "^5.7.2"

package/src/config.ts

@@ -14,11 +14,21 @@ export interface ShellBgSettings {
   autoBackgroundMs: number;
   /** Bytes of the log tail shown in a status/collect result. */
   tailBytes: number;
+  /**
+   * How many jobs may be alive at once before a request to background one
+   * more is refused and an auto-background transition is skipped. Every live
+   * job is a whole process tree plus an open log; a session that keeps
+   * starting servers and watchers should hear about it rather than
+   * accumulate them without bound. Only the backgrounding decision is capped
+   * — a command itself is never refused.
+   */
+  maxBackground: number;
 }
 
 export const DEFAULT_SETTINGS: ShellBgSettings = {
   autoBackgroundMs: 30_000,
   tailBytes: 64 * 1024,
+  maxBackground: 8,
 };
 
 const LIMITS: Record<keyof ShellBgSettings, { min: number; max: number }> = {
@@ -26,6 +36,7 @@ const LIMITS: Record<keyof ShellBgSettings, { min: number; max: number }> = {
   // does not make every command look long-running.
   autoBackgroundMs: { min: 0, max: 3_600_000 },
   tailBytes: { min: 1024, max: 4 * 1024 * 1024 },
+  maxBackground: { min: 1, max: 64 },
 };
 
 export function resolveSettings(

package/src/env.ts

@@ -0,0 +1,67 @@
+/**
+ * The environment a backgrounded command is spawned with.
+ *
+ * pi's own bash does not hand the child a raw `process.env`: it starts from
+ * `getShellEnv()` (process.env with `<agentDir>/bin` — where pi auto-installs
+ * `fd`/`rg` — prepended to PATH) and, since exposeSessionEnvironment defaults on,
+ * sets PI_SESSION_ID / PI_SESSION_FILE / PI_PROVIDER / PI_MODEL /
+ * PI_REASONING_LEVEL from the session ctx. The pi host never sets those PI_*
+ * vars in its own process.env (it injects them only into its bash tool's child),
+ * so a command spawned with plain `process.env` sees none of them and cannot
+ * resolve the managed `fd`/`rg` — despite the inherited description/guidelines
+ * promising both. This rebuilds the same env so a backgrounded command behaves
+ * like a foreground one.
+ *
+ * Pure and pi-free (the binDir and a minimal ctx shape are passed in) so it unit
+ * tests standalone. Zero dependencies — node:path only.
+ */
+import { delimiter } from "node:path";
+
+/** The slice of the pi ExtensionContext this needs, kept structural for tests. */
+export interface EnvCtx {
+  sessionManager?: { getSessionId?(): string; getSessionFile?(): string | undefined };
+  model?: { provider?: string; id?: string } | undefined;
+  thinkingLevel?: string | undefined;
+}
+
+/**
+ * Copy `base`, prepend `binDir` to PATH when absent, and set the PI_* session
+ * variables from `ctx`. The ctx reads are wrapped so a stale ctx after a
+ * `/reload` degrades to plain env rather than failing the spawn.
+ */
+export function buildEnv(
+  ctx: EnvCtx,
+  binDir: string,
+  base: NodeJS.ProcessEnv = process.env,
+): NodeJS.ProcessEnv {
+  const env: NodeJS.ProcessEnv = { ...base };
+
+  // PATH is case-insensitive on Windows; find the real key so we do not create a
+  // second, ignored "PATH" alongside an inherited "Path". Only prepend when the
+  // bin dir is not already on it (mirrors pi's getShellEnv hasBinDir check).
+  if (binDir) {
+    const pathKey = Object.keys(env).find((k) => k.toLowerCase() === "path") ?? "PATH";
+    const current = env[pathKey] ?? "";
+    const entries = current.split(delimiter).filter(Boolean);
+    if (!entries.includes(binDir)) {
+      env[pathKey] = [binDir, current].filter(Boolean).join(delimiter);
+    }
+  }
+
+  try {
+    const sid = ctx.sessionManager?.getSessionId?.();
+    if (sid) env.PI_SESSION_ID = sid;
+    const sessionFile = ctx.sessionManager?.getSessionFile?.();
+    if (sessionFile) env.PI_SESSION_FILE = sessionFile;
+    const model = ctx.model;
+    if (model) {
+      if (model.provider) env.PI_PROVIDER = model.provider;
+      if (model.id) env.PI_MODEL = model.id;
+    }
+    if (ctx.thinkingLevel) env.PI_REASONING_LEVEL = ctx.thinkingLevel;
+  } catch {
+    // Stale ctx (e.g. read after /reload): a plain env still runs the command.
+  }
+
+  return env;
+}

package/src/format.ts

@@ -4,6 +4,7 @@
  */
 import type { Job } from "./types.ts";
 import { readTail, countLines } from "./tail.ts";
+import { sanitizeOutput } from "./sanitize.ts";
 
 function secs(ms: number): string {
   if (ms < 1000) return `${ms}ms`;
@@ -21,11 +22,13 @@ export function header(job: Job): string {
   const verdict =
     job.status === "running"
       ? "running"
-      : job.signal
-        ? `signal ${job.signal}`
-        : job.status === "killed"
-          ? "killed"
-          : `exit ${job.exitCode ?? "?"}`;
+      : job.status === "orphaned"
+        ? "orphaned (another session)"
+        : job.signal
+          ? `signal ${job.signal}`
+          : job.status === "killed"
+            ? "killed"
+            : `exit ${job.exitCode ?? "?"}`;
   return `[${job.id} · ${job.status} · ${verdict} · ${duration(job)}]`;
 }
 
@@ -36,14 +39,17 @@ export function header(job: Job): string {
  */
 export function formatResult(job: Job, tailBytes: number): string {
   const tail = readTail(job.logPath, tailBytes);
+  // Cleaned the way pi's own bash cleans its output for the model; the log on
+  // disk (which the truncation note points at) keeps every byte.
+  const text = sanitizeOutput(tail.text);
   const lines = [header(job)];
-  if (tail.text.trim() === "") {
+  if (text.trim() === "") {
     lines.push(job.status === "running" ? "(no output yet)" : "(no output)");
   } else {
     if (tail.truncated) {
-      lines.push(`… showing the last ${countLines(tail.text)} lines — full log: ${job.logPath}`);
+      lines.push(`… showing the last ${countLines(text)} lines — full log: ${job.logPath}`);
     }
-    lines.push(tail.text.replace(/\n+$/, ""));
+    lines.push(text.replace(/\n+$/, ""));
   }
   return lines.join("\n");
 }

package/src/pending.ts

@@ -61,9 +61,9 @@ export function backgroundedResult(input: BackgroundedInput): BackgroundedResult
         "with no id lists everything still running.",
       ]
     : [
-        "This is a headless run: nothing is delivered after your turn ends. Call",
-        `${input.collectWith} with id "${input.id}" again in this same turn until it reports finished —`,
-        "do not end your turn expecting the result to arrive on its own.",
+        "This is a headless run: nothing is delivered after your turn ends. Collect it",
+        `in this same turn — ${input.collectWith} {id: "${input.id}", wait: 60} blocks until it`,
+        "finishes (or the wait elapses); do not end your turn expecting the result to arrive on its own.",
       ];
   return {
     text: [head, line, "", ...tail].join("\n"),

package/src/registry.ts

@@ -16,7 +16,7 @@ import { join } from "node:path";
 import type { Job } from "./types.ts";
 import { isRecord } from "./types.ts";
 
-function isAlive(pid: number | null): boolean {
+export function isAlive(pid: number | null | undefined): boolean {
   if (!pid || pid <= 0) return false;
   try {
     process.kill(pid, 0);
@@ -47,6 +47,11 @@ export class JobRegistry {
     mkdirSync(join(baseDir, "logs"), { recursive: true });
   }
 
+  /** The directory this registry's sidecars and logs live under. */
+  dir(): string {
+    return this.baseDir;
+  }
+
   logPathFor(id: string): string {
     return join(this.baseDir, "logs", `${id}.log`);
   }
@@ -58,6 +63,7 @@ export class JobRegistry {
       command,
       cwd,
       pid: null,
+      hostPid: process.pid,
       status: "running",
       exitCode: null,
       signal: null,
@@ -94,7 +100,28 @@ export class JobRegistry {
     }
   }
 
-  /** Load persisted jobs and settle any whose process has since died. */
+  /** Read one job's sidecar from disk, or null if missing/corrupt. */
+  readSidecar(id: string): Job | null {
+    try {
+      const raw = JSON.parse(readFileSync(join(this.baseDir, `${id}.json`), "utf8"));
+      return isJob(raw) ? raw : null;
+    } catch {
+      return null;
+    }
+  }
+
+  /**
+   * Load persisted jobs and reconcile any still marked running:
+   *
+   *  - a record written by this same host process (a `/reload` keeps the host
+   *    pid) is a genuine survivor — kept running if its pid is still alive so the
+   *    new instance can adopt it, settled to `done` if the process has since died;
+   *  - a record from any other host pid (another session that reused this dir, or
+   *    a crashed one, or a pre-upgrade sidecar with no hostPid) is *not* ours: its
+   *    pid may since belong to something unrelated, so we never treat it as
+   *    running (which would make session_shutdown kill a stranger's pid) — it is
+   *    surfaced as `orphaned` and kept out of running().
+   */
   load(): void {
     let files: string[];
     try {
@@ -108,9 +135,14 @@ export class JobRegistry {
         const raw = JSON.parse(readFileSync(join(this.baseDir, f), "utf8"));
         if (!isJob(raw)) continue;
         const job = raw;
-        if (job.status === "running" && !isAlive(job.pid)) {
-          job.status = "done";
-          job.endedAt = job.endedAt ?? Date.now();
+        if (job.status === "running") {
+          if (job.hostPid !== process.pid) {
+            job.status = "orphaned";
+            job.endedAt = job.endedAt ?? Date.now();
+          } else if (!isAlive(job.pid)) {
+            job.status = "done";
+            job.endedAt = job.endedAt ?? Date.now();
+          }
         }
         this.jobs.set(job.id, job);
         const n = Number(job.id.replace(/^bg-/, ""));

package/src/sanitize.ts

@@ -0,0 +1,50 @@
+/**
+ * The same cleaning pi's own bash tool applies to what the model sees
+ * (bash-executor.ts: `sanitizeBinaryOutput(stripAnsi(text)).replace(/\r/g, "")`).
+ *
+ * This package re-registers `bash`, and a job's log is captured raw on disk on
+ * purpose (the detached spawn writes straight to a file; no in-process pipes).
+ * Without this step the tail handed back for a chatty build or dev server is
+ * escape codes and carriage-return progress frames — tokens the native tool
+ * would never have spent. The on-disk log stays byte-for-byte; only the text
+ * that reaches the model is cleaned. Zero dependencies: pi does not export
+ * these helpers, so the regex (ansi-regex, MIT) and the code-point filter are
+ * vendored here verbatim.
+ */
+
+// Valid string terminator sequences are BEL, ESC\, and 0x9c
+const ST = "(?:\\u0007|\\u001B\\u005C|\\u009C)";
+// OSC sequences only: ESC ] ... ST (non-greedy until the first ST)
+const OSC = `(?:\\u001B\\][\\s\\S]*?${ST})`;
+// CSI and related: ESC/C1, optional intermediates, optional params (supports ; and :) then final byte
+const CSI = "[\\u001B\\u009B][[\\]()#;?]*(?:\\d{1,4}(?:[;:]\\d{0,4})*)?[\\dA-PR-TZcf-nq-uy=><~]";
+const ANSI = new RegExp(`${OSC}|${CSI}`, "g");
+
+export function stripAnsi(text: string): string {
+  // Fast path: ANSI codes require ESC (7-bit) or CSI (8-bit) introducer.
+  if (!text.includes("\u001B") && !text.includes("\u009B")) return text;
+  return text.replace(ANSI, "");
+}
+
+/**
+ * Drop characters that crash string-width or corrupt a terminal: control
+ * characters (except tab/newline/CR), Unicode format characters, lone
+ * surrogates and undefined code points.
+ */
+export function sanitizeBinaryOutput(text: string): string {
+  return Array.from(text)
+    .filter((char) => {
+      const code = char.codePointAt(0);
+      if (code === undefined) return false;
+      if (code === 0x09 || code === 0x0a || code === 0x0d) return true;
+      if (code <= 0x1f) return false;
+      if (code >= 0xfff9 && code <= 0xfffb) return false;
+      return true;
+    })
+    .join("");
+}
+
+/** Exactly what pi's bash gives the model: no ANSI, no control noise, no `\r`. */
+export function sanitizeOutput(text: string): string {
+  return sanitizeBinaryOutput(stripAnsi(text)).replace(/\r/g, "");
+}

package/src/types.ts

@@ -3,7 +3,7 @@
  * No imports from pi packages: src/ typechecks and unit-tests standalone.
  */
 
-export type JobStatus = "running" | "done" | "failed" | "killed";
+export type JobStatus = "running" | "done" | "failed" | "killed" | "orphaned";
 
 export interface Job {
   /** Short session-monotonic id, e.g. "bg-1". */
@@ -12,6 +12,14 @@ export interface Job {
   cwd: string;
   /** OS pid of the shell process; null before spawn or if spawn failed. */
   pid: number | null;
+  /**
+   * pid of the pi host that spawned this job. A record whose hostPid is not the
+   * current process was written by another (or a since-crashed) host: its pid is
+   * not ours to signal, so it is never treated as running here. A `/reload`
+   * keeps the same host pid, so genuine reload survivors still adopt. Optional
+   * so a pre-upgrade sidecar (no hostPid) is simply treated as foreign.
+   */
+  hostPid?: number;
   status: JobStatus;
   /** Process exit code, once finished. */
   exitCode: number | null;