@sebastianandreasson/pi-autonomous-agents 0.5.0 → 0.5.1

package/README.md

@@ -1,58 +1,79 @@
-# PI Harness
+# PI Autonomous Agents
 
-`pi-harness` is a portable CLI/workflow package for running a local PI-based unattended loop with:
+`@sebastianandreasson/pi-autonomous-agents` is an npm package for running a bounded unattended [PI](https://pi.dev/) workflow inside another repository.
 
-- a `developer` pass
-- a fast verification step
-- a skeptical `tester` pass
-- optional periodic multimodal visual review
-- tester-owned final commit by default
+It orchestrates:
 
-The package is intentionally generic. It does not know how to navigate or test a specific app on its own.
+- a `developer` turn
+- a fast local verification step
+- an independent `tester` turn
+- an optional focused `developerFix` turn when verification/tester finds a real issue
+- optional periodic visual review from screenshots
 
-## What Belongs In The Package
+The package is intentionally generic. It handles supervision, prompts, runtime state, telemetry, retries, and guardrails. The consuming repo still owns its own tasks, instructions, tests, model endpoints, and screenshot capture flow.
 
-- supervisor/orchestration
-- PI adapter/runtime integration
+## Install
+
+```bash
+npm install -D @sebastianandreasson/pi-autonomous-agents
+```
+
+Then in the consuming repo, tell your agent:
+
+```text
+Find SETUP.md in @sebastianandreasson/pi-autonomous-agents and set everything up for this repository.
+```
+
+The package ships a top-level [SETUP.md](./SETUP.md) specifically for that workflow.
+
+## What This Package Owns
+
+- unattended loop orchestration
+- PI adapter integration
 - config loading
-- telemetry
-- loop guards, timeout guards, and retries
-- tester feedback + visual feedback handoff
-- optional legacy harness git finalize step for `commitMode: "plan"`
-- multimodal visual review client
+- prompt assembly
+- verification/tester/visual-review handoff
+- timeout and loop guards
+- telemetry and run summaries
+- runtime isolation and stale-run recovery
 
-## What Stays Per Project
+## What Each Repo Must Provide
 
 - `TODOS.md`
-- project instructions
-- browser tests
-- visual capture flow
-- app-specific verification commands
-- app/server startup scripts
+- repo-specific `pi/DEVELOPER.md`
+- repo-specific `pi/TESTER.md`
+- a fast bounded `testCommand`
+- model configuration that actually matches the local/cloud providers in use
+- optionally a screenshot capture command for visual review
 
-## Layout
+## Quick Start In A Repo
+
+The normal setup shape is:
 
 ```text
-packages/pi-harness/
-  package.json
-  pi.config.json
-  templates/DEVELOPER.md
-  templates/TESTER.md
-  docs/PI_SUPERVISOR.md
-  src/
-    cli.mjs
-    pi-client.mjs
-    pi-config.mjs
-    pi-prompts.mjs
-    pi-repo.mjs
-    pi-report.mjs
-    pi-rpc-adapter.mjs
-    pi-supervisor.mjs
-    pi-telemetry.mjs
-    pi-visual-once.mjs
-    pi-visual-review.mjs
+TODOS.md
+pi.config.json
+pi/
+  DEVELOPER.md
+  TESTER.md
+```
+
+Typical scripts:
+
+```json
+{
+  "scripts": {
+    "pi:mock": "PI_CONFIG_FILE=pi.config.json PI_TRANSPORT=mock PI_TEST_CMD= pi-harness once",
+    "pi:once": "PI_CONFIG_FILE=pi.config.json pi-harness once",
+    "pi:run": "PI_CONFIG_FILE=pi.config.json pi-harness run",
+    "pi:report": "PI_CONFIG_FILE=pi.config.json pi-harness report",
+    "pi:visual:once": "PI_CONFIG_FILE=pi.config.json pi-harness visual-once"
+  }
+}
 ```
 
+Start from [templates/pi.config.example.json](./templates/pi.config.example.json), [templates/DEVELOPER.md](./templates/DEVELOPER.md), [templates/TESTER.md](./templates/TESTER.md), and [templates/gitignore.fragment](./templates/gitignore.fragment).
+
 ## CLI
 
 ```bash
@@ -65,65 +86,212 @@ pi-harness adapter
 pi-harness visual-review-worker
 ```
 
-Use `PI_CONFIG_FILE` to point the harness at a project-local config file. If you do not provide one, the bundled generic `pi.config.json` is used as a fallback.
-
-## Setup In Another Repo
-
-After installing the package:
+Use `PI_CONFIG_FILE` to point at the repo-local config file:
 
 ```bash
-npm install -D @sebastianandreasson/pi-autonomous-agents
+PI_CONFIG_FILE=pi.config.json pi-harness once
 ```
 
-you can tell another agent in that repo:
-
-```text
-Find SETUP.md in @sebastianandreasson/pi-autonomous-agents and set everything up for this repository.
+If `PI_CONFIG_FILE` is not set, the package falls back to the bundled generic [pi.config.json](./pi.config.json).
+
+## Core Workflow
+
+Each real iteration works like this:
+
+1. `developer` implements one unchecked task from `TODOS.md`.
+2. The harness runs the configured fast verification command.
+3. If verification passes, `tester` reviews the change independently.
+4. If tester or verification fails, the findings go back to `developerFix` for one focused repair pass.
+5. If tester reaches `PASS`, tester creates the final commit directly by default.
+6. Every `N` successful iterations, optional visual review can inspect screenshots and veto the success if it finds a real problem.
+
+The default commit model is `commitMode: "agent"`. The older harness-managed parsed commit-plan flow still exists as `commitMode: "plan"`, but it is now a compatibility mode rather than the default.
+
+## Recommended Model Setup
+
+The package supports:
+
+- one default text model via `piModel`
+- one default visual-review model via `visualReviewModel`
+- optional per-role overrides via `roleModels`
+- per-model endpoint config in `models`
+
+Typical pattern:
+
+- local model for `developer`
+- local model for `developerRetry`
+- local model for `developerFix`
+- local or slightly stronger model for `tester`
+- stronger frontier model only for `visualReview`
+
+Example:
+
+```json
+{
+  "piModel": "local/text-model",
+  "visualReviewModel": "cloud/vision-model",
+  "models": {
+    "local/text-model": {
+      "baseUrl": "http://localhost:8000/v1",
+      "apiKey": "local",
+      "vision": false
+    },
+    "local/tester-model": {
+      "baseUrl": "http://localhost:8000/v1",
+      "apiKey": "local",
+      "vision": false
+    },
+    "cloud/vision-model": {
+      "baseUrl": "https://api.openai.com/v1",
+      "apiKeyEnv": "OPENAI_API_KEY",
+      "vision": true
+    }
+  },
+  "roleModels": {
+    "developer": "local/text-model",
+    "developerRetry": "local/text-model",
+    "developerFix": "local/text-model",
+    "tester": "local/tester-model",
+    "visualReview": "cloud/vision-model"
+  }
+}
 ```
 
-The package ships a top-level [SETUP.md](./SETUP.md) specifically for that workflow.
+Important:
+
+- do not guess model ids
+- if using a custom OpenAI-compatible provider, verify `<baseUrl>/models`
+- if using PI models directly, verify `pi --list-models`
+- if `PI_CODING_AGENT_DIR` points at a repo-local PI home, make sure it is bootstrapped and contains `models.json`
+
+The harness now preflights those checks before starting a real run.
 
-If you want to wipe all harness-generated state and start over cleanly in a repo, run:
+## Important Config Fields
 
-```bash
-PI_CONFIG_FILE=pi.config.json pi-harness clear-history
-```
+Common fields in `pi.config.json`:
+
+- `taskFile`
+- `developerInstructionsFile`
+- `testerInstructionsFile`
+- `transport`
+- `adapterCommand`
+- `piModel`
+- `models`
+- `roleModels`
+- `commitMode`
+- `promptMode`
+- `testCommand`
+- `visualReviewEnabled`
+- `visualCaptureCommand`
+- `continueAfterSeconds`
+- `toolContinueAfterSeconds`
+- `noEventTimeoutSeconds`
+- `toolNoEventTimeoutSeconds`
+- `largeFileWarningLines`
+- `largeSpecWarningLines`
+
+Key defaults:
+
+- `transport`: `adapter`
+- `commitMode`: `agent`
+- `promptMode`: `compact`
+- `piTools`: `read,edit,write,find,ls,bash`
+- `continueAfterSeconds`: `300`
+- `toolContinueAfterSeconds`: `900`
+- `noEventTimeoutSeconds`: `900`
+- `toolNoEventTimeoutSeconds`: `1800`
+
+## Prompt and Tooling Behavior
+
+The package is optimized for local models by default:
+
+- prompts are compacted before handoff
+- changed-file lists and feedback excerpts are capped
+- prompts prefer `read` for source inspection
+- shell is intended for `git`, tests, and narrow diagnostics
+- the adapter warns on obvious oversized shell-based file reads
+- the supervisor emits large-file/spec warnings when touched files are getting risky
+
+This is deliberate. Large monolith files, huge e2e specs, and broad TODO items are one of the main causes of local-model drift and retry loops.
+
+Recommended repo shape:
+
+- keep TODO items very small and implementation-shaped
+- split giant stores/modules before they become constant edit hotspots
+- split ever-growing end-to-end specs into scenario files
+- keep the default `testCommand` to a bounded smoke check, not a multi-minute happy-path run
+
+## Runtime Isolation And Recovery
 
-The command removes configured harness history/runtime files and verifies that no configured history paths remain afterward.
+Recent versions of the package isolate each run more aggressively:
 
-For prompt debugging, the harness also writes the exact assembled prompt for the current role to `.pi-last-prompt.txt` by default.
-For flow debugging, it also writes a machine-readable `.pi-last-iteration.json` summary with the selected task, tester verdict, commit-plan state, and terminal reason.
-For run isolation, the supervisor also maintains `.pi-runtime/active-run.json` and stores PI sessions plus per-run telemetry under `.pi-runtime/runs/<runId>/`.
+- active ownership lock at `.pi-runtime/active-run.json`
+- per-run runtime directory under `.pi-runtime/runs/<runId>/`
+- per-run PI sessions and telemetry
+- `runId` added to telemetry
+- in-progress iteration state persisted before agent work starts
+- stale run locks recovered when the owning PID is gone
+- timeout cleanup kills the full spawned process group, not only the direct child
 
-## Generic Contracts
+That is meant to prevent orphaned timed-out agents or concurrent supervisors from corrupting shared state.
 
-- `taskFile`: usually `TODOS.md`
-- `developerInstructionsFile`: per-project developer instructions
-- `testerInstructionsFile`: per-project tester instructions
-- `roleModels`: optional per-role model overrides
-- `commitMode`: `agent` by default, `plan` only for legacy harness-managed commit parsing
-- `promptMode`: `compact` by default
-- `testCommand`: fast verification command
-- `visualCaptureCommand`: project-defined screenshot capture command
-- `visualFeedbackFile`: latest visual-review handoff
-- `testerFeedbackFile`: latest tester-review handoff
+## Debugging Artifacts
 
-For unattended loops, keep `testCommand` fast and bounded, such as a smoke suite. Long real-time Playwright happy-path specs belong in an explicit nightly or post-run lane, not the default developer/tester inner loop.
+Useful files during a run:
 
-Keep TODO items extremely small and implementation-shaped when using weaker local models. Broad tasks tend to produce much longer turns, more retries, and more tester drift than narrow one-step tasks.
+- `.pi-last-prompt.txt`
+  Exact assembled prompt for the current role.
+- `.pi-last-output.txt`
+  Latest agent output snapshot.
+- `.pi-last-verification.txt`
+  Latest verification output snapshot.
+- `.pi-last-iteration.json`
+  Structured summary of the last completed iteration.
+- `.pi-state.json`
+  Persistent harness state, including in-progress iteration data.
+- `pi.log`
+  Main run log.
+- `pi_telemetry.jsonl`
+- `pi_telemetry.csv`
+- `.pi-runtime/active-run.json`
+- `.pi-runtime/runs/<runId>/...`
 
-The adapter heartbeat is PI-RPC-event based. Streaming shell output does not count as progress on its own, so long-running tools should rely on the tool-aware watchdog thresholds rather than terminal streaming.
+`pi-harness report` summarizes recent telemetry and surfaces things like terminal reasons and large-file warnings.
 
-The supervisor now enforces single-run ownership per repo/config. If a stale run crashed mid-iteration, the next run recovers the unfinished iteration number from `.pi-state.json` instead of silently rolling forward.
+## Visual Review Contract
 
-`piModel` remains the default text model, but you can override specific roles with `roleModels` such as `developer`, `developerRetry`, `developerFix`, `tester`, and `visualReview`. `testerCommit` is only relevant if you opt back into `commitMode: "plan"`.
+Visual review is optional and generic. The harness does not know how to navigate your app.
 
-By default, successful tester passes should stage and create the commit directly in the same PI turn. The old commit-plan parsing flow is still available as `commitMode: "plan"`, but it is now a compatibility mode rather than the default.
+If enabled, your repo must provide a real screenshot capture command that writes a manifest under the configured capture directory. The manifest shape is documented in [docs/PI_SUPERVISOR.md](./docs/PI_SUPERVISOR.md).
 
-Prompt/context handoff is compact by default. The harness now caps prior feedback excerpts, changed-file lists, verification excerpts, and prompt note handoff. If needed, tune `maxPromptChangedFiles`, `maxVisualFeedbackLines`, `maxTesterFeedbackLines`, `maxPromptNotesLines`, and `maxVerificationExcerptLines`.
+Visual review should be used as a periodic audit, not as the default inner-loop gate.
 
-The default coding tool mix is now safer for local models: `read,edit,write,find,ls,bash`. Prompts explicitly steer source inspection toward `read` and reserve shell usage for `git`, tests, and narrow diagnostics.
+## Resetting Harness State
 
-The harness also emits lightweight large-file warnings for touched source/spec files and carries them into `.pi-last-iteration.json`, `pi-harness report`, and relevant prompts. Tune `largeFileWarningLines` and `largeSpecWarningLines` if needed.
+If you want to wipe harness-generated state and start fresh:
+
+```bash
+PI_CONFIG_FILE=pi.config.json pi-harness clear-history
+```
+
+That clears configured harness runtime/history artifacts and verifies they are gone. It does not remove project source files.
+
+## Docs
+
+- [SETUP.md](./SETUP.md)
+  Agent-facing setup instructions for consuming repos.
+- [docs/PI_SUPERVISOR.md](./docs/PI_SUPERVISOR.md)
+  More detailed flow, adapter, and runtime documentation.
+- [templates/PROJECT_SETUP.md](./templates/PROJECT_SETUP.md)
+  Minimal consuming-repo layout summary.
+
+## Development
+
+In this package repo:
+
+```bash
+npm run check
+npm test
+```
 
-The harness expects screenshot capture to produce a `manifest.json` plus image files under the configured visual capture directory.
+The package requires Node `>=20`.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@sebastianandreasson/pi-autonomous-agents",
   "private": false,
-  "version": "0.5.0",
+  "version": "0.5.1",
   "type": "module",
   "description": "Portable unattended PI harness for developer/tester/visual-review loops.",
   "license": "MIT",