pi-task-dispatcher 1.0.0 → 3.0.0

package/README.md

@@ -1,133 +1,192 @@
 # pi-task-dispatcher
 
-Delegate bounded mechanical and engineering subtasks to configurable Pi worker models without replacing the model selected for the main Pi session.
+Supervised mechanical and engineering delegations for Pi.
 
-## How it works
+Each delegation runs in an isolated, in-memory Pi RPC session. After every cohesive tool batch, a worker-side gate stops execution at a safe boundary and returns a bounded state report. The manager can then inspect the result and explicitly continue, revise, compact, checkpoint, cancel, or take over.
 
-Your selected Pi model remains the **orchestrator**: it decides whether to delegate, supplies a self-contained task, evaluates the worker report, reviews material changes, and owns the final answer. This package adds two model-only tools:
+## Features
 
-| Tool | Use it when |
-| --- | --- |
-| `delegate_mechanical` | The outcome and constraints are precise and the work is bounded, multi-step, and primarily inspection, exact edits, repetitive transformation, commands, retrieval, or verification. Use a direct tool for a single trivial operation. |
-| `delegate_engineering` | A self-contained engineering subtask needs investigation, implementation, debugging, refactoring, testing, review, research, or analysis. Include the outcome, constraints, and acceptance conditions. |
-
-Both workers operate in the current workspace and can modify files when their configured tools permit it.
-
-After the orchestrator selects a delegation tool, its role is routed deterministically to that role's configured `provider/model`. That routing does **not** make a worker's output, changes, or correctness deterministic; treat every worker report as input to review and verification.
+- **Two worker roles:** mechanical execution and broader engineering work.
+- **Enforced supervision:** a private parent-child IPC gate calls `ctx.abort()` after persisted tool results; pausing does not depend on the worker following a prompt.
+- **Explicit control:** continue, revise, recalculate, compact, checkpoint, cancel, or take over.
+- **Resumable context:** continuation uses the same in-memory worker session and completed tool results.
+- **Least privilege:** every start selects an exact Pi built-in worker-tool allowlist; delegation tools and unrelated discovered extensions are unavailable to the worker.
+- **Model agnostic:** both roles inherit Pi's active model and thinking level unless an explicit role override is configured.
+- **Formula-owned budgets:** the manager supplies semantic workload units, while the dispatcher calculates turn and time ceilings from workload, role, and delegated-model limits.
+- **Context insight:** boundary reports include worker context usage, compactions, turns, elapsed time, action counts, and recommendations.
+- **Recoverable reports:** the complete action ledger and worker text remain in structured tool details; bounded model-facing text retains recent actions plus both the beginning and ending of long reports.
+- **Deterministic UI:** tool renderers display task, current activity, elapsed time, context pressure, and terminal success/failure colors. A compact paused-state line appears in Pi's footer.
 
 ## Requirements
 
-- Pi 1.0.0 or later
+- Pi 1.0.3 or later
 - Node.js 22.19.0 or later
-- A configured, authenticated Pi model for each worker role you invoke
+- One active, authenticated Pi model; optional role-specific models must also be available and authenticated
 
-## Install
+The supervision gate requires the parent Pi process and child Pi CLI to run under Node.js with an inherited IPC channel. The dispatcher fails closed when that boundary cannot be established; it does not fall back to prompt-only supervision. Worker processes disable ordinary extension discovery and load only the generated supervision gate.
 
-Install the published package for your Pi user configuration:
+## Installation
+
+Install the published package for the current Pi user:
 
 ```sh
-pi install npm:pi-task-dispatcher@1.0.0
+pi install npm:pi-task-dispatcher
 ```
 
-For a local checkout, run this from its parent directory:
+## Tools
 
-```sh
-pi install ./pi-task-dispatcher
-```
+| Tool | Responsibility |
+| --- | --- |
+| `delegate_mechanical` | Start a supervised mechanic delegation. |
+| `delegate_engineering` | Start a supervised engineer delegation. |
+| `delegate_control` | Apply one explicit manager decision to a paused delegation. |
+| `delegate_status` | Recover the current or most recent state without changing it. |
 
-To register either source in the current project's `.pi/settings.json` instead, add `--local`:
+Only one live delegation is currently permitted. A terminal delegation may be inspected until a new one replaces it.
 
-```sh
-pi install npm:pi-task-dispatcher@1.0.0 --local
-```
+## Starting a delegation
+
+Both role tools require:
 
-Project packages load only after that project is trusted. In an already running Pi session, run:
+- `task`: a self-contained objective, scope, constraints, and acceptance checks;
+- `tools`: the smallest exact allowlist selected from Pi's built-in tools and active for the main agent;
+- `workload`: semantic estimates used by the deterministic budget formula.
 
-```text
-/reload
+```json
+{
+  "task": "Inspect the parser cache, correct its invalidation, and run the focused tests.",
+  "tools": ["read", "edit", "bash"],
+  "workload": {
+    "investigationUnits": 3,
+    "changeUnits": 2,
+    "verificationUnits": 2,
+    "expectedLongRunningSeconds": 60
+  }
+}
 ```
 
-`/reload` reloads extensions and other discovered resources. The worker configuration below is read again for every delegation.
+A workload unit is intentionally semantic:
 
-## Default workers
+- `investigationUnits`: cohesive components or evidence groups to inspect;
+- `changeUnits`: related edit groups;
+- `verificationUnits`: focused check or test groups;
+- `expectedLongRunningSeconds`: additional expected command time.
 
-| Role | Model | Thinking | Timeout | Turn limit |
-| --- | --- | --- | ---: | ---: |
-| Mechanic | `openai-codex/gpt-5.6-luna` | `max` | 300 seconds | 20 |
-| Engineer | `openai-codex/gpt-5.6-terra` | `max` | 900 seconds | 30 |
+At least one investigation, change, or verification unit is required.
 
-By default, both workers receive this explicit tool allowlist:
+Isolated workers accept `read`, `bash`, `powershell`, `edit`, `write`, `grep`, `find`, and `ls` when the selected tools are also active in the main Pi session. Extension tools are not supported because the child disables extension discovery.
 
-```text
-read,bash,edit,write
-```
+The current formula derives planned work turns from those units, applies a role factor and a bounded scale based on the selected delegated model's own context window, reserves two additional turns for synthesis or recovery, then derives active execution time from calculated turns plus expected long-running time. Administrative ceilings always win. Formula inputs and results are returned in every state report so they can be reviewed and recalculated. Time spent paused for manager deliberation does not consume the active execution budget.
+
+## Safe action boundaries
+
+A worker run proceeds as follows:
+
+1. Pi produces one assistant response.
+2. Its complete tool-call batch runs and tool results are persisted.
+3. The worker-only gate reports the boundary through private IPC.
+4. The gate aborts the automatic next worker turn.
+5. The RPC run settles while the child process and in-memory context remain alive.
+6. The delegation tool returns the new state to the manager.
+
+An action batch can contain parallel tool calls from one worker response. Already-running tools are allowed to finish. The boundary prevents another productive model turn from starting without a manager decision.
+
+## Controlling a paused delegation
 
-`pi-task-dispatcher` provides only these four native Pi tools by default; it does not provide `web_search`, `fetch_content`, `get_search_content`, or `source_check`. If you have the separate `pi-web-access` package installed, you may explicitly add those tools to a role's `tools` array:
+`delegate_control` accepts these actions:
+
+| Action | Effect |
+| --- | --- |
+| `continue` | Resume from completed results. `actionBatches` may lease 1–10 batches before returning. |
+| `revise` | Add controlling instructions, then resume. |
+| `recalculate` | Supply remaining semantic workload and extend the current ceilings from observed turns and active execution time without running the worker. |
+| `compact` | Compact the idle worker's own context and return refreshed state. |
+| `take_over` | Stop the delegation externally and transfer remaining work to the manager. |
+| `checkpoint` | Stop externally and preserve a synthetic checkpoint from observed state. |
+| `cancel` | Stop externally and close the job. |
+
+Example:
 
 ```json
 {
-  "mechanic": {
-    "tools": ["read", "bash", "edit", "write", "web_search", "fetch_content", "get_search_content", "source_check"]
-  }
+  "jobId": "engineer-2",
+  "action": "continue",
+  "actionBatches": 2
 }
 ```
 
-## Configure workers
+The dispatcher enforces takeover, cancellation, budget exhaustion, no-progress policy, and process cleanup. These actions do not depend on worker cooperation.
+
+## State reports and screen output
+
+A paused result contains:
+
+- a session-local sequential identity such as `engineer-2`, plus role, model, and status;
+- the delegated task and live command or tool activity while execution is running;
+- boundary sequence and current phase;
+- completed, failed, and potentially mutating action counts;
+- recent bounded activity labels;
+- active elapsed time and turns against calculated budgets;
+- context tokens, context window, percentage, and compaction counts;
+- a recommended next decision;
+- bounded worker text, if any.
+
+The TUI renders the same structured state as a compact block. Expanding the tool result reveals bounded worker text. The complete action ledger and worker text remain in result details for state reconstruction rather than being discarded by display truncation.
+
+When paused, one unindented footer status appears below Pi's primary report. It shows the identity, active elapsed time, last activity, and context as percentage/window—for example, `10.3%/32k context`. The percentage and window both come from that delegation's selected model and live worker-session statistics, not from the main model.
+
+## Worker compaction
 
-Optionally create `~/.pi/agent/task-dispatcher.json`. If Pi's agent directory is overridden, use `<agent-dir>/task-dispatcher.json` instead. Missing role objects and fields inherit the defaults.
+The dispatcher explicitly enables automatic compaction in every worker session. Pi's automatic compaction remains the overflow safety mechanism.
 
-This is a complete valid configuration using the defaults:
+The dispatcher also:
+
+- observes compaction lifecycle events;
+- accounts for reported compaction usage;
+- queries `get_session_stats` at safe boundaries;
+- recommends proactive compaction when context usage reaches the current policy threshold;
+- permits explicit worker compaction only while paused.
+
+Worker compaction is independent from compaction of the main Pi conversation.
+
+## Configuration
+
+The optional configuration file is `task-dispatcher.json` in Pi's agent directory.
 
 ```json
 {
-  "version": 1,
-  "mechanic": {
-    "model": "openai-codex/gpt-5.6-luna",
-    "thinking": "max",
-    "tools": [
-      "read",
-      "bash",
-      "edit",
-      "write"
-    ],
-    "timeoutSeconds": 300,
-    "maxTurns": 20
-  },
-  "engineer": {
-    "model": "openai-codex/gpt-5.6-terra",
-    "thinking": "max",
-    "tools": [
-      "read",
-      "bash",
-      "edit",
-      "write"
-    ],
-    "timeoutSeconds": 900,
-    "maxTurns": 30
+  "policy": {
+    "absoluteMaxTurns": 96,
+    "absoluteMaxSeconds": 7200,
+    "pausedLeaseSeconds": 1800,
+    "noProgressSeconds": 600
   }
 }
 ```
 
-| Field | Meaning |
-| --- | --- |
-| `version` | Optional. When present, it must be `1`. |
-| `mechanic`, `engineer` | Optional per-role objects. Each supplied field overrides that role's default. |
-| `model` | A `provider/model` string. Pi verifies that the exact model exists in its registry and has usable authentication before starting the worker. |
-| `thinking` | One of `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, or `max`. |
-| `tools` | A non-empty array of tool names. It replaces, rather than extends, the default allowlist; duplicate names are removed. |
-| `timeoutSeconds` | Positive integer wall-clock limit for the worker process. |
-| `maxTurns` | Positive integer limit for a continuing tool-using worker loop. |
+Configuration is read for every new delegation. By default, both roles inherit the manager's active model and thinking level, so installation assumes no particular provider or model access. A role may optionally override `model` with `"provider/model"`, `thinking`, or both. Policy values are administrative failsafes, not ordinary task budgets.
 
-## Isolation, limits, and safety
+Version 3 intentionally ignores the version 2 fields `maxTurns`, `timeoutSeconds`, and `checkpointGraceSeconds`. Remove them from local configuration after migration.
 
-Each delegation starts a separate, no-session Pi child process in the current workspace. The worker receives its role prompt, selected model, thinking level, and only the configured `tools` allowlist. The dispatcher does not register either delegation tool inside worker processes, so workers cannot recursively delegate through this package.
+## Lifecycle and cleanup
 
-The parent cancellation signal is propagated to the child. Workers are stopped on timeout, and an ongoing tool-using loop is stopped at its configured turn limit. While a worker runs, its tool activity can be reported as progress. Its final tool result includes the role, model, thinking level, configured tool names, status, turns, duration, exit information, activity, and aggregated usage; non-completed runs are returned as errors with a diagnostic and any available partial report. Reported failure states include configuration, model, and process errors, cancellation, timeout, turn-limit, and incomplete outcomes.
+- A parent-operation abort stops the active worker stage.
+- A paused lease eventually closes an abandoned worker.
+- Session shutdown or extension reload disposes the child and runtime-controlled temporary files.
+- Worker process failures reject pending RPC operations and produce a failed state.
+- Worker role instructions and the supervision gate exist only in a runtime-controlled temporary directory.
+- The worker uses `--no-session`; no worker session file is persisted.
+
+Process separation is not a security sandbox. Worker tools execute with the Pi host's operating-system permissions. Tool availability does not establish task scope or authorization.
+
+## Development checks
+
+```sh
+npm run check
+npm run test:gate
+```
 
-> **Security warning:** Worker process isolation is not a security sandbox. A worker can use every tool in its configured allowlist with the Pi host's permissions in the current environment. Use the smallest practical allowlist and delegate only work you are prepared to authorize.
+`check` runs static release invariants and deterministic fake-RPC lifecycle and renderer scenarios. The fake child verifies pause/resume, usage deltas, full action-ledger retention, intercepted prompts, missing-boundary failure, fail-closed timeouts, pre-aborted operations, partial argument rendering, stable repeated rendering, and manager-state transitions without a model request.
 
-## Short workflow
+`test:gate` performs a small live model integration check with the active parent model exported by Pi through `PI_PROVIDER`, `PI_MODEL`, and `PI_REASONING_LEVEL`, falling back to Pi's configured default when those values are unavailable; it contains no provider or model assumption. Optional `PI_TASK_DISPATCHER_GATE_MODEL` and `PI_TASK_DISPATCHER_GATE_THINKING` overrides are available for explicit test matrices. The probe verifies that the private gate pauses after a read batch and that the same in-memory child can subsequently complete. It requires configured credentials and therefore is not a hermetic unit test.
 
-1. Select the normal Pi model that should orchestrate the task.
-2. Give it a bounded request, for example: “Use `delegate_engineering` to diagnose the failing parser test. Do not change public APIs; run the focused test; return the cause, files changed, and test result.”
-3. The orchestrator calls the selected worker tool, receives its report, inspects relevant changes and verification, then produces the final response.