pi-tian-background-terminals 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tian Zuo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,124 @@
+# pi-tian-background-terminals
+
+A managed replacement for Pi's built-in `bash` tool.
+
+Every model shell command follows one path: start it, wait briefly, and return
+its final output if it finishes. If it outlives the initial wait, return control
+to the model while the command continues as a session-scoped background
+terminal. Its final result is delivered automatically exactly once.
+
+```text
+■ 2 background terminals running • /ps to view
+```
+
+## Canonical `bash` override
+
+Pi officially supports replacing a built-in tool by registering the same name.
+This extension registers `bash`, so the model no longer chooses between normal
+Bash and a separate background tool. Pi may display an expected startup warning
+that the built-in has been overridden.
+
+Parameters:
+
+- `command` — Bash script to execute.
+- `timeout` — optional hard total runtime timeout in seconds. When reached, the
+  whole process tree is terminated. There is no default runtime timeout.
+- `working_dir` — optional working directory; defaults to the current directory.
+- `title` — optional short `/ps` label; defaults to a bounded one-line command.
+- `yield_time_ms` — optional initial wait, default **10 seconds**, range
+  **250–30,000 ms**.
+
+Behavior:
+
+1. Resolve Bash using Pi's normal platform logic and preserve Pi's configured
+   `shellPath` and `shellCommandPrefix` settings.
+2. Inject the same `PI_SESSION_ID`, `PI_SESSION_FILE`, `PI_PROVIDER`, `PI_MODEL`,
+   and `PI_REASONING_LEVEL` environment values as built-in Bash.
+3. Start the command with no interactive stdin and stream bounded progress in
+   the normal Bash tool row.
+4. If it exits during `yield_time_ms`, return its final status and bounded
+   head+tail output. Non-zero exits and hard timeouts are Bash tool errors.
+5. If it remains alive, return an id such as `bt-1`. The model should continue
+   working rather than poll. A follow-up message wakes it exactly once when the
+   process exits.
+
+There are no model-facing status, list, kill, polling, or stdin tools. The user
+owns inspection and termination through `/ps`.
+
+### Safe foreground fallback
+
+If the managed Effect runtime cannot initialize—or `start()` returns a typed
+spawn error proving no child was created—the extension falls back to Pi's
+standard foreground Bash implementation. The result includes a warning that
+automatic yielding and `/ps` tracking were unavailable.
+
+The extension deliberately never retries through fallback after a spawn,
+non-zero exit, timeout, or abort. Re-executing an arbitrary shell command could
+duplicate destructive side effects.
+
+## `/ps` viewer
+
+While at least one terminal runs, a one-line widget renders above the editor.
+`/ps` opens a two-stage full-screen overlay:
+
+1. **List** — every tracked terminal; `↑/↓`/`j`/`k` select, `Enter` inspect,
+   `x` stop the selected running terminal, `Esc` close.
+2. **Detail** — metadata, `t`-toggled stdout/stderr, live tailing, scrolling
+   (`↑/↓`, `PgUp/PgDn`, `g`/`G`), and `x` to stop.
+
+## Design
+
+- **Automatic yielding, no polling.** Quick commands return directly; only
+  commands that outlive the initial wait become background work. Completion
+  uses `pi.sendMessage(..., { deliverAs: "followUp", triggerTurn: true })`.
+- **Exactly-once completion.** A race-safe waiter token decides whether the
+  initial Bash call or the later follow-up owns settlement. A drain-once map
+  handles delivery retries without duplicates.
+- **Bounded head+tail memory plus full capture.** Each stdout/stderr stream
+  retains a stable **256 KiB startup head** and rolling tail within a **2 MiB**
+  cap. Omitted middle bytes are marked. Complete output spills from byte zero
+  to an owner-only file (`0600` in a `0700` session directory), capped at
+  256 MiB per stream.
+- **Separate stdout and stderr.** Both streams are independently retained,
+  spilled, inspected, and formatted.
+- **No interactive stdin.** Normal commands see EOF. The legacy WSL Bash
+  transport may receive the script over stdin, but that pipe is closed
+  immediately and cannot be used interactively.
+- **Process-tree termination.** POSIX children use their own process group;
+  Windows uses `taskkill /T`. Shutdown and hard timeouts send SIGTERM and
+  escalate to SIGKILL after two seconds.
+- **Session scoped.** `/new`, `/resume`, `/fork`, `/reload`, and quit terminate
+  every process tree and remove spill files.
+
+The async core uses [Effect](https://effect.website) v4. Node
+`child_process` output remains callback-driven. See
+[`docs/implementation-guide.md`](./docs/implementation-guide.md) for internal
+invariants.
+
+## Install
+
+```bash
+pi install npm:pi-tian-background-terminals
+```
+
+Restart Pi or run `/reload` afterwards.
+
+## Development
+
+This workspace uses Effect v4 and TypeScript 7 (`tsgo`), so it is checked in
+isolation:
+
+```bash
+npm install -w pi-tian-background-terminals
+cd packages/pi-background-terminals
+npm run check
+npm test
+```
+
+## Credits
+
+Ported from [`davis7dotsh/my-pi-setup`](https://github.com/davis7dotsh/my-pi-setup/tree/main/extensions/background-terminals).
+
+## License
+
+MIT

package/docs/implementation-guide.md

@@ -0,0 +1,430 @@
+# Background terminals implementation guide
+
+This document describes the invariants behind `pi-tian-background-terminals`.
+The package overrides Pi's built-in `bash` model tool and adds one user command,
+`/ps`.
+
+## 1. Product contract
+
+Every model shell command goes through `bash`:
+
+1. Validate the command, working directory, yield wait, and optional hard
+   timeout.
+2. Resolve Bash and Pi's shell settings.
+3. Spawn with no interactive stdin and capture stdout/stderr separately.
+4. Wait for a bounded initial yield period (default 10 seconds, accepted range
+   250–30,000 ms).
+5. If the process settles during that wait, return its final state and output in
+   the Bash result. A non-zero exit, kill, or timeout is a tool error.
+6. Otherwise return a terminal id and leave the process running.
+7. When a yielded process settles, deliver its final result as an exactly-once
+   follow-up that wakes the model.
+
+The model has no status, list, kill, poll, or stdin tools. It should continue
+working after a command yields. The user inspects and stops running terminals
+with `/ps`.
+
+A process is session-scoped. It never survives `/new`, `/resume`, `/fork`,
+`/reload`, or quit.
+
+## 2. Why override `bash`
+
+Pi combines built-in and extension tools by name; an extension definition wins
+when it registers the same name. Registering `bash` therefore replaces the
+built-in execution implementation while keeping one canonical shell tool in the
+model schema.
+
+The public schema keeps built-in compatibility:
+
+```text
+command        required string
+timeout        optional hard total runtime timeout in seconds
+```
+
+It adds managed-execution controls:
+
+```text
+working_dir    optional working directory
+title          optional /ps label
+yield_time_ms  optional initial wait, 250–30,000 ms
+```
+
+The override supplies its own prompt snippet and guidelines because Pi does not
+inherit prompt metadata from built-ins. It omits custom tool renderers so Pi's
+built-in Bash call/result renderers are inherited. Successful results use
+`details: undefined`, which is a valid `BashToolDetails` shape. Separate full-log
+paths remain in the textual stdout/stderr sections.
+
+## 3. Safe fallback boundary
+
+A fallback may only occur before it is possible for a command to have run.
+
+`getManager()` resolves and wires the Effect service before `manager.start()`.
+If that resolution fails, the broken runtime is disposed and a fresh
+`createBashToolDefinition(cwd, settings)` executes the command in Pi's standard
+foreground Bash implementation.
+
+`SpawnError` also carries `fallbackSafe`. It is true only for synchronous shell
+resolution or `spawn()` failures where the manager can prove no child was
+created. Those errors may use the same fallback without resetting an otherwise
+healthy manager. Shutdown races set it false. The result is always prefixed with
+a warning that background yielding and `/ps` were unavailable.
+
+No fallback occurs after:
+
+- `start()` creates a child;
+- spawn succeeds or emits an asynchronous error;
+- a command exits non-zero;
+- a hard timeout;
+- an abort;
+- the managed concurrency cap is reached.
+
+Retrying after any of those points could execute arbitrary side effects twice.
+A failed command is a command result, not evidence that another executor should
+run it again.
+
+## 4. Shell compatibility
+
+The tool preserves Pi's normal Bash behavior:
+
+- `SettingsManager` resolves trusted global/project `shellPath` and
+  `shellCommandPrefix` settings.
+- `getShellConfig(shellPath)` selects Bash using Pi's cross-platform rules:
+  Bash on POSIX and Git Bash/MSYS/Cygwin-compatible Bash on Windows, with Pi's
+  normal fallback behavior.
+- `shellCommandPrefix` is prepended only to the script sent to Bash. The
+  snapshot retains the exact model-supplied command.
+- The tool reconstructs the same live session environment values:
+  `PI_SESSION_ID`, `PI_SESSION_FILE`, `PI_PROVIDER`, `PI_MODEL`, and
+  `PI_REASONING_LEVEL`. Inherited stale values are removed first.
+
+Most shells receive the script as a `-c` argument and have stdin ignored.
+Legacy WSL Bash may require a one-shot stdin script transport; the extension
+writes the script and closes the pipe immediately. Neither path provides an
+interactive input surface.
+
+## 5. File layout
+
+```text
+index.ts                    Pi boundary, bash override, fallback, /ps
+src/domain.ts               Snapshot/status/error types
+src/manager.ts              Effect service and process lifecycle
+src/output.ts               Bounded head+tail stream retention
+src/prompt.ts               Tool metadata and model-facing formatting
+src/result-delivery.ts      Drain-once completion delivery map
+src/runtime.ts              ManagedRuntime and Effect→Promise boundary
+src/ui/output-view.ts       ANSI-safe wrapped output rendering
+src/ui/ps.ts                /ps list and detail overlays
+```
+
+The manager owns mutable lifecycle state. Consumers receive readonly live
+`TerminalSnapshot` objects whose stdout/stderr properties are getters over
+`OutputBuffer`.
+
+## 6. Domain model
+
+A terminal moves exactly once from `running` to one final status:
+
+```text
+running ── exit 0 ─────────────────────────────► done
+running ── non-zero exit / process error ──────► failed
+running ── hard runtime deadline ──────────────► timed_out
+running ── /ps stop / session teardown ────────► killed
+```
+
+A snapshot includes:
+
+- stable `bt-N` id;
+- exact model command, bounded title, absolute cwd, and pid;
+- timestamps, optional timeout, and final exit code or signal;
+- separate stdout/stderr `OutputView` values;
+- bounded lifecycle/spill error text.
+
+`OutputView` exposes:
+
+- `head`: stable startup prefix;
+- `tail`: rolling recent suffix;
+- `text`: head + omission marker + tail for `/ps`;
+- `totalBytes` and `truncatedBytes`;
+- optional complete spill-file path.
+
+## 7. Execution flow
+
+`index.ts` first performs side-effect-free validation. It preserves the exact
+script text and trims only to reject an empty command and derive a title.
+Filesystem stat errors and non-directory paths are reported before manager
+resolution.
+
+Manager execution has two calls:
+
+```text
+manager.start(...)
+manager.waitForSettlement(id, yield_time_ms)
+```
+
+`start` is uninterruptible between `spawn()` and registry insertion. This avoids
+an abort window where a live child exists without a manager entry or scope.
+
+During the initial wait, a per-terminal subscription emits bounded progress
+updates at most every 100 ms. Exceptions thrown by the display callback are
+ignored; presentation cannot affect process execution.
+
+The initial wait is abortible. Aborting it does not kill the process. The error
+identifies the terminal id, and eventual settlement remains eligible for an
+automatic follow-up.
+
+The returned result has two forms:
+
+- **Final:** status/output are returned directly and any deferred completion for
+  the tiny start→wait race is consumed. Failed, killed, and timed-out final
+  states throw so Pi marks the Bash result as an error.
+- **Yielded:** status is `running`, the id and captured startup output are
+  returned, and later settlement becomes a follow-up.
+
+## 8. Hard runtime timeout
+
+`timeout` is independent from `yield_time_ms`:
+
+- `yield_time_ms` controls only when the model regains control;
+- `timeout` is a total process lifetime and terminates the process tree.
+
+The accepted timeout upper bound is Node's maximum timer delay,
+2,147,483,647 ms, matching Pi's built-in Bash limit. There is no default hard
+timeout.
+
+The manager installs an unref'ed timer only after the entry is registered. If
+the deadline wins before a natural `exit`, it marks the entry `timed_out` and
+closes the entry scope. Natural exit metadata that already won is not rewritten
+merely because descendants still hold inherited output pipes open.
+
+Every settlement and disposal path clears the timer. Session teardown clears
+all timers before closing scopes so a shutdown is reported as `killed`, not as
+a coincidental timeout.
+
+## 9. Yield/settlement linearization
+
+The most important race is:
+
+```text
+initial yield wins  <──►  process settlement wins
+```
+
+Returning both the Bash result and an automatic follow-up would duplicate the
+same completion. Returning neither would lose it.
+
+`waitForSettlement` registers a waiter token in `settlementWaiters` before
+racing the terminal's `Deferred` against `Effect.sleep(yield_time_ms)`.
+
+When settlement wins:
+
+1. `settle` marks every registered waiter token `consumed = true`.
+2. It computes the settle-hook `consumed` flag.
+3. It completes the terminal `Deferred`.
+4. It invokes the settle hook.
+5. The Bash call returns the final snapshot.
+
+When yield wins:
+
+1. The timeout branch synchronously removes its waiter token.
+2. It returns the still-running snapshot.
+3. A later settlement sees no waiter and therefore queues a follow-up.
+
+The synchronous token removal is the linearization point. Cleanup is also an
+Effect finalizer, so an interrupted wait cannot leave stale interest that
+suppresses a future completion.
+
+There is a second defensive layer in `index.ts`: when a final snapshot is
+returned, `resultDelivery.consume(id)` removes any result deferred during the
+small gap between `start` and waiter registration.
+
+## 10. Exactly-once follow-up delivery
+
+The settle hook receives `(snapshot, consumed)`:
+
+- `consumed = true`: the initial Bash wait or an internal kill operation is
+  already returning the final state; do not queue a follow-up.
+- `consumed = false`: copy the final snapshot into the drain-once delivery map.
+
+The map is keyed by terminal id. `drain()` clears entries before delivery, so a
+result can be sent only once. A failed `pi.sendMessage` call re-defers the same
+id for a later `agent_settled` retry.
+
+Delivery uses:
+
+```ts
+pi.sendMessage(message, {
+  deliverAs: "followUp",
+  triggerTurn: true,
+});
+```
+
+A busy agent receives the message after its current run settles. An idle agent
+is woken immediately. No model-driven polling is required.
+
+## 11. Head+tail output retention
+
+Each stdout/stderr stream has a 2 MiB in-memory cap:
+
+```text
+stable head:   256 KiB
+rolling tail:  remaining 1.75 MiB
+```
+
+`OutputBuffer.push` performs these steps:
+
+1. Send the complete decoded chunk to the spill callback before retention.
+2. Fill the stable head until its budget is reached.
+3. Seal the head as soon as any byte extends past that budget.
+4. Append remaining bytes to the tail.
+5. Evict bytes from the front of the tail until it fits its budget.
+6. Cut only at UTF-8 code point boundaries.
+7. Compute omitted bytes as `total - retainedHead - retainedTail`.
+
+Bounded slices are copied so a small retained head/tail cannot pin a giant
+source Buffer. Once sealed, the head never changes. This preserves startup
+configuration and first errors while the tail tracks recent logs.
+
+`OutputView.text` inserts an explicit marker when the middle was omitted:
+
+```text
+<startup head>
+... 7340032 bytes omitted ...
+<recent tail>
+```
+
+The marker itself is not counted against the retained-byte cap.
+
+### Model-facing truncation
+
+The 2 MiB view is still too large for model context. `prompt.ts` applies Pi's
+truncation utilities again with smaller per-result budgets, allocating one
+quarter to startup head and three quarters to recent tail.
+
+Initial Bash result budgets:
+
+```text
+stdout: 16 KiB / 400 lines
+stderr:  8 KiB / 200 lines
+```
+
+Automatic completion budgets:
+
+```text
+stdout:  8 KiB / 40 lines
+stderr:  4 KiB / 20 lines
+```
+
+Every model-visible path remains bounded even if a child is a firehose.
+
+## 12. Spill files and backpressure
+
+Before spawning output consumers, the manager creates a private per-session
+temporary directory (`0700`). Each stream writes to a separate `0600` file from
+its first byte.
+
+Spills are capped at 256 MiB per stream. If the cap or an I/O error is reached,
+the full-log pointer is cleared and a bounded `errorText` note is attached.
+
+When `WriteStream.write()` reports backpressure, the matching child stream is
+paused and resumed on `drain`. In-memory retention still receives the triggering
+chunk; subsequent child output is flow-controlled by Node.
+
+Settlement waits for spill streams to flush before publishing the final
+snapshot. The flush wait is bounded so a broken filesystem cannot leave a
+terminal permanently running.
+
+## 13. Process lifecycle and termination
+
+Children use separate stdout/stderr pipes and no interactive stdin. On POSIX,
+`detached: true` gives the child its own process group. Windows uses `taskkill
+/T`, with `/F` for force kill.
+
+Termination is:
+
+```text
+SIGTERM whole tree
+wait up to 2 seconds for close
+SIGKILL whole tree
+bounded final close/settle wait
+```
+
+Exit metadata is recorded on Node's `exit` event, but settlement occurs on
+`close` so output has reached EOF. If descendants inherit the pipes and hold
+them open after the shell exits, bounded cleanup closes the entry scope and
+reaps the process group.
+
+The entry scope is the single cleanup path for `/ps` stop, hard timeout,
+pruning, internal kill calls, and runtime disposal.
+
+## 14. Capacity and pruning
+
+At most eight terminals may run concurrently. Start reserves a slot
+synchronously before spawning, so parallel tool calls cannot race through the
+cap. Reaching the cap is an explicit error; it does not bypass management via
+the foreground fallback.
+
+The registry retains at most 32 live/settled entries for `/ps`. When over the
+limit, it removes the oldest settled entries; running entries are never pruned.
+Small tombstones retain final kill-report facts across pruning races.
+
+## 15. `/ps` UI
+
+The manager exposes a synchronous `TerminalReadModel` for TUI rendering:
+
+- `list`, `get`, and `size`;
+- global and per-id subscriptions;
+- fire-and-forget user kill;
+- settle-hook registration.
+
+The list overlay supports selection and stopping. The detail overlay provides:
+
+- command metadata;
+- stdout/stderr toggle;
+- ANSI/control sanitization at render time;
+- wrapped output with cached layouts keyed by `(buffer version, width)`;
+- live tail pinning and scrollback.
+
+## 16. Session teardown
+
+`session_shutdown` clears UI/delivery state and disposes the ManagedRuntime.
+The manager finalizer runs `disposeAll`, which:
+
+1. marks the manager disposed;
+2. removes every registry entry;
+3. clears every runtime timeout;
+4. closes every entry scope concurrently with bounds;
+5. waits for detached cleanup fibers within the shutdown bound;
+6. removes the private spill directory.
+
+The `disposed` check after process setup closes a child that raced with a
+teardown sweep, preventing an unregistered survivor.
+
+## 17. Required tests
+
+The package test suite covers:
+
+- registration of only the `bash` override plus `/ps`;
+- quick completion without a duplicate follow-up;
+- yielded completion with exactly one automatic delivery;
+- safe pre-spawn foreground fallback;
+- non-zero commands executing only once, without fallback retry;
+- Pi session environment and command-prefix preservation;
+- bounded streaming updates during the initial wait;
+- hard timeout status and tree termination;
+- Bash-specific syntax on the resolved shell;
+- abort leaving eventual completion deliverable;
+- process-tree kill and SIGKILL escalation;
+- session disposal and spill-directory cleanup;
+- output head stability, rolling tail, UTF-8 boundaries, and omission counts;
+- complete spill capture beyond the memory cap;
+- drain-once result delivery;
+- `/ps` selection, sanitization, wrapping, and cache behavior.
+
+Validation commands:
+
+```bash
+npm run check --workspace pi-tian-background-terminals
+npm test --workspace pi-tian-background-terminals
+npm run typecheck
+npm pack --dry-run --workspace pi-tian-background-terminals
+```