@tduyng/pi-shell 0.0.1 → 0.0.3

package/README.md

@@ -1,58 +1,64 @@
 # @tduyng/pi-shell
 
-Background shell execution for pi. One `bg` tool covers the full lifecycle: start, read output, kill, and list tasks.
+Park a long command in the background and get woken when it matters. One `bg` tool covers start, read, kill, and list.
 
-## The bg tool
+## What it costs
 
-A single tool with an `action` field. Each `bg` task runs in the current working directory with a fresh shell, like pi's own bash. There is no persistent shell session.
+`bg` is 197 tokens a request for the whole lifecycle, where four tools would have cost four schemas.
+
+The saving is in the wait, not the schema. A test suite, a build, or a dev server run for minutes and then return a log nobody reads whole. This tool holds the bytes and hands the model the tail it asked for, so a five minute build does not become a five minute stall in the transcript.
+
+## The tool
+
+One tool with an `action` field. Each task runs in the current working directory with a fresh shell, like pi's own bash. There is no persistent shell session.
 
 ### start
 
-Spawns a detached command. Returns `task_id` (8-char UUID prefix), `pid`, and the absolute `log` path. stdout and stderr stream to a log file at `<agentDir>/shell-logs/<taskId>.log`. The task runs silently until you ask about it. Optional `timeout` caps the task (max 600 seconds). Default is no timeout.
+Spawns a detached command and returns `task_id` (an 8-character UUID prefix), `pid`, and the absolute log `path`. stdout and stderr stream to `<agentDir>/shell-logs/<taskId>.log`. The task runs silently until you ask about it. An optional `timeout` caps it at 600 seconds, with no timeout by default.
 
 ### output
 
-Reads new bytes since the last read using a byte cursor. The `wait` parameter controls behavior:
+Reads new bytes since the last read, through a byte cursor.
 
-- `"now"`: returns immediately with whatever is available.
-- `"next_write"`: parks the agent until the first new byte appears, then wakes it with the new output.
-- `"exit"`: parks the agent until the process ends, then delivers the verdict plus the unread tail.
+| `wait` | Behaviour |
+| --- | --- |
+| `now` | Return whatever is available |
+| `next_write` | Park the agent until the first new byte, then wake it |
+| `exit` | Park the agent until the process ends, then deliver the verdict and the unread tail |
 
-The cursor never advances on wake reads, so a later `bg` output call still sees every byte from the stored offset onward.
+A wake read does not advance the cursor, so a later `output` still sees every byte from the stored offset.
 
 ### kill
 
-Sends SIGTERM to the process group, polls every 50ms for up to 1 second, then escalates to SIGKILL. Accepts `task_id`, raw `pid`, or `port` (resolved to a pid via `lsof -ti`).
+SIGTERM to the process group, polled every 50ms for up to a second, then SIGKILL. Takes a `task_id`, a raw `pid`, or a `port` resolved through `lsof -ti`.
 
 ### status
 
-Lists all tasks with id, pid, state (running, exited, killed), cursor offset, and the first 80 characters of the command.
-
-## Honest exits
+Every task with id, pid, state (running, exited, killed), cursor offset, and the first 80 characters of the command.
 
-Pi 0.99 already reports a signal-killed command as a failure. Its bash backend returns `128 + signal` for a child that died without an exit code, and raises when a command ends with neither an exit code nor a signal. This extension does not override that backend.
+## Waking only when asked
 
-`shell-process.test.ts` runs `kill -9` against pi's own bash tool and asserts a nonzero result in the 128+n family, so a pi release that reintroduces the silent success fails here.
+Only a task the model parked on, through `wait: "exit"` or `wait: "next_write"`, injects a message when it finishes. A burst of exits coalesces into one message on a byte budget: 8 KB total by default, at least 1 KB per task. The wake carries the exit verdict, the unread tail, and the log path, so the model gets the result without a read-back.
 
-A `tool_result` handler annotates the two cases where the exit code alone cannot tell the model what happened. An exit 1 from `grep`, `rg`, or `find` means no matches, and a command killed by its timeout did not finish cleanly. Each note stops the model from retrying a search that already succeeded.
+The wake fires from the `onExit` callback through `queueMicrotask` rather than polling. A wake that lands mid-turn goes to the follow-up queue and is delivered at the next `agent_settled`.
 
-## Middle-out output view
+A task parked on `wait: "exit"` also gets a silence check. Every 30 seconds it reads the log size, and after 10 minutes with no new bytes the agent is woken once with the elapsed time and the log path. It fires again only after output resumes and stops again. No model call runs for the check.
 
-Large logs are truncated to a head and tail, defaulting to 8KB total, with an exact omitted-byte count in between. The read window per side is 16KB, split on UTF-8 sequence boundaries so characters are never cut mid-byte. The full log stays on disk at the path always shown in results.
+## Reading a log without drowning in it
 
-## Wake on exit
+Large output is cut to a head and a tail, 8 KB total by default, with the exact omitted byte count between them. Each side reads up to 16 KB, split on UTF-8 sequence boundaries so a character is never cut mid-byte. The full log stays on disk at the path every result shows.
 
-Only a task the model parked on (via `wait: "exit"` or `wait: "next_write"`) injects a user message when it finishes. A burst of exits coalesces into one message sharing a byte budget: default 8KB total, minimum 1KB per task. The wake carries the exit verdict, the unread tail, and the full log path per task, so the model gets the result without a read-back.
+A `tool_result` handler annotates the two cases where an exit code cannot say what happened: exit 1 from `grep`, `rg`, or `find` means no matches, and a command killed by its timeout did not finish cleanly. Each note stops the model retrying a search that already succeeded.
 
-The wake fires from the `onExit` callback via `queueMicrotask`, not from polling. If the agent is mid-turn, the wake lands in the follow-up queue and is delivered at the next `agent_settled` boundary.
+pi already reports a signal-killed command as a failure: its bash backend returns `128 + signal` and raises when a command ends with neither an exit code nor a signal. `shell-process.test.ts` runs `kill -9` against pi's own bash tool and asserts a nonzero result in that family, so a pi release that reintroduces the silent success fails here.
 
-## Footer indicator
+## Footer
 
-The footer shows a dot and the live task count (e.g. "2 bg running") while any background task is active. It clears when the last task exits.
+A dot and the live count while any task runs, for example `2 bg running`. It clears when the last one exits.
 
-## Task lifecycle
+## Lifecycle
 
-Each `bg` task is in-memory only. It dies with the pi process. Persisted session entries record the command, log path, and start time so the model can read the log after a reload. The live process handle is gone. Detached grandchildren that inherit the shell's stdout keep writing to the log after the shell exits. The log stream stays open until both the stdout and stderr pipes close. Ending it on the first close would drop data the second pipe still delivers.
+Each task is in-memory and dies with the pi process. The session records the command, log path, and start time, so the model can still read the log after a reload, even though the process handle is gone. Detached grandchildren that inherit the shell's stdout keep writing after the shell exits, and the log stream closes only when both pipes close, since ending it on the first close would drop data the second one still delivers.
 
 ## Install
 
@@ -64,4 +70,4 @@ Local checkout: `pi install /path/to/dotfiles/pi/agent/extensions/pi-shell`
 
 ## Configure
 
-- `SHELL`: set to `0`, `false`, `no`, or `off` to disable the extension entirely.
+`SHELL`, set to `0`, `false`, `no`, or `off`, disables the extension.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@tduyng/pi-shell",
-  "version": "0.0.1",
-  "description": "Background processes the agent parks on: one bg tool with start/output/kill/status, cursor reads that never re-read bytes, and a runtime wake-up when a task exits.",
+  "version": "0.0.3",
+  "description": "One bg tool for background processes, woken on exit, with cursor reads that never re-read bytes.",
   "keywords": [
     "pi",
     "pi-package",

package/src/index.ts

@@ -16,6 +16,7 @@ import {
     killTask,
     middleOut,
     benignExitNote,
+    isNoMatchCommand,
     pidForPort,
     readNewBytes,
     readUnreadTail,
@@ -23,6 +24,7 @@ import {
     spawnTask,
     watchForWrite
 } from './process.js'
+import { watchStall } from './stall.js'
 import type { TaskRecord } from './process.js'
 const isString = new Validator(t.string()).isValidObject
 
@@ -95,7 +97,7 @@ export async function buildWakeText(records: TaskRecord[], budget = WAKE_TAIL_MA
 }
 
 export async function sendBackgroundWake(pi: Pick<ExtensionAPI, 'sendMessage'>, text: string): Promise<void> {
-    pi.sendMessage(
+    await pi.sendMessage(
         { customType: WAKE_TYPE, content: text, display: true },
         { deliverAs: 'followUp', triggerTurn: true }
     )
@@ -116,8 +118,7 @@ async function flushWakes(pi: ExtensionAPI): Promise<boolean> {
 
 function isBenignNoMatch(rec: Pick<TaskRecord, 'command' | 'exitCode' | 'signal'>): boolean {
     if (rec.exitCode !== 1 || rec.signal !== null) return false
-    const cmd = rec.command.trim()
-    return /^(grep|rg|find)\b/.test(cmd) || /\b(grep|rg)\b/.test(cmd)
+    return isNoMatchCommand(rec.command)
 }
 
 interface PersistedTask {
@@ -226,6 +227,14 @@ async function bgAction(pi: ExtensionAPI, ctx: ExtensionContext, params: BgInput
             )
         }
 
+        if (!rec.wakeRequested) {
+            watchStall(rec, (silentMs) => {
+                void sendBackgroundWake(
+                    pi,
+                    `bg task ${params.task_id} has written no output for ${Math.round(silentMs / 60000)} min and is still running. Check its log, or kill it if it is stuck. ${LOG_PREFIX}${rec.logPath}`
+                ).catch(() => undefined)
+            })
+        }
         rec.wakeRequested = true
         return textResult(
             `Watching task ${params.task_id} (bytes ${read.start}-${read.end} of ${read.size}). ` +

package/src/process.ts

@@ -1,7 +1,7 @@
 import { spawn } from 'node:child_process'
 import type { ChildProcess } from 'node:child_process'
 import { randomUUID } from 'node:crypto'
-import { createWriteStream, mkdirSync, readdirSync, statSync, unlinkSync, watch } from 'node:fs'
+import { appendFileSync, createWriteStream, mkdirSync, readdirSync, statSync, unlinkSync, watch } from 'node:fs'
 import type { WriteStream } from 'node:fs'
 import { open, stat } from 'node:fs/promises'
 import { basename, join } from 'node:path'
@@ -131,6 +131,7 @@ export function spawnTask(options: SpawnOptions): TaskRecord {
     mkdirSync(logDirPath, { recursive: true })
     sweepTaskLogs(logDirPath, Date.now())
     const logPath = join(logDirPath, `${taskId}.log`)
+    appendFileSync(logPath, '')
     const stream: WriteStream = createWriteStream(logPath, { flags: 'a' })
 
     const shell = resolveShell()
@@ -381,6 +382,7 @@ export function watchForWrite(
     let pollTimer: ReturnType<typeof setInterval> | undefined
     let settled = false
     let checking: Promise<boolean> | undefined
+    let pending = false
 
     const stopPolling = () => {
         if (pollTimer) clearInterval(pollTimer)
@@ -394,7 +396,10 @@ export function watchForWrite(
     }
 
     const check = (): Promise<boolean> => {
-        if (checking) return checking
+        if (checking) {
+            pending = true
+            return checking
+        }
         checking = (async (): Promise<boolean> => {
             if (settled) return false
             try {
@@ -409,6 +414,10 @@ export function watchForWrite(
             }
         })().finally(() => {
             checking = undefined
+            if (pending && !settled) {
+                pending = false
+                void check()
+            }
         })
         return checking
     }
@@ -434,9 +443,7 @@ export function watchForWrite(
 
     const exitBridge = () => {
         if (settled) return
-        void (checking ?? Promise.resolve(false))
-            .then(() => check())
-            .then(finishWithoutOutput)
+        void (checking ?? Promise.resolve(false)).then(() => check()).then(finishWithoutOutput)
     }
     rec.exitWaiters.add(exitBridge)
 
@@ -472,7 +479,7 @@ export function benignExitNote(
     raisedText: string
 ): string | undefined {
     if (!isError) return undefined
-    if (exitCode === 1 && /\b(?:grep|rg)\b/.test(command)) {
+    if (exitCode === 1 && isNoMatchCommand(command)) {
         return '\n[Note: exit 1 here means no matches found, not an error.]'
     }
     if (exitCode === undefined && /^Command timed out after\b/.test(raisedText.trimEnd().split('\n').at(-1) ?? '')) {
@@ -481,6 +488,10 @@ export function benignExitNote(
     return undefined
 }
 
+export function isNoMatchCommand(command: string): boolean {
+    return /^(?:grep|rg)\b[^|&;\n]*$/.test(command.trim())
+}
+
 export function middleOut(text: string, maxBytes: number): MiddleOutView {
     const total = Buffer.byteLength(text, 'utf8')
     if (total <= maxBytes) {

package/src/stall.ts

@@ -0,0 +1,46 @@
+import { statSync } from 'node:fs'
+
+import { registry } from './process.js'
+import type { TaskRecord } from './process.js'
+
+export const STALL_MS = 10 * 60 * 1000
+const STALL_CHECK_MS = 30 * 1000
+
+export interface StallState {
+    size: number
+    since: number
+    reported: boolean
+}
+
+export interface StallStep {
+    next: StallState
+    report: boolean
+}
+
+export function advanceStall(prev: StallState, size: number, now: number): StallStep {
+    if (size !== prev.size) return { next: { size, since: now, reported: false }, report: false }
+    if (prev.reported || now - prev.since < STALL_MS) return { next: prev, report: false }
+    return { next: { ...prev, reported: true }, report: true }
+}
+
+export function watchStall(rec: TaskRecord, onStall: (silentMs: number) => void): void {
+    let state: StallState = { size: -1, since: Date.now(), reported: false }
+    const timer = setInterval(() => {
+        const live = registry.get(rec.taskId)
+        if (!live || live.exitCode !== null || live.signal !== null) {
+            clearInterval(timer)
+            return
+        }
+        let size: number
+        try {
+            size = statSync(rec.logPath).size
+        } catch {
+            return
+        }
+        const now = Date.now()
+        const step = advanceStall(state, size, now)
+        state = step.next
+        if (step.report) onStall(now - state.since)
+    }, STALL_CHECK_MS)
+    timer.unref()
+}